Conference Paper

Deriving Validity Time in Knowledge Graph

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

Knowledge Graphs (KGs) are a popular means to represent knowledge on the Web, typically in the form of node/edge labelled directed graphs. We consider temporal KGs, in which edges are further annotated with time intervals, reflecting when the relationship between entities held in time. In this paper, we focus on the task of predicting time validity for unannotated edges. We introduce the problem as a variation of relational embedding. We adapt existing approaches, and explore the importance example selection and the incorporation of side information in the learning process. We present our experimental evaluation in details.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... However, recent researches in TKGs focus on temporal link prediction, which is simply single-hop. The methods include tensor decomposition methods [8-10] timestamp transformation methods [11][12][13][14], etc. A complex logical query involving multiple facts for multi-hop reasoning is not fully explored yet. ...
... Transformation-based methods use timestamps in temporal facts to transform embeddings of entity/relation in the facts and then use a score function to evaluate the validity of the facts. Existing approaches include relation transformation [11] and entity-relation transformation [12][13][14]. However, transformation-based methods are only confined to the one-hop link prediction task and cannot answer a multi-hop query that involves multiple entities and relations in the knowledge graph. ...
... if answering timestamps (11) where · 1 is the L 1 norm. ...
Preprint
Full-text available
Multi-hop logical reasoning over knowledge graph (KG) plays a fundamental role in many artificial intelligence tasks. Recent complex query embedding (CQE) methods for reasoning focus on static KGs, while temporal knowledge graphs (TKGs) have not been fully explored. Reasoning over TKGs has two challenges: 1. The query should answer entities or timestamps; 2. The operators should consider both set logic on entity set and temporal logic on timestamp set. To bridge this gap, we define the multi-hop logical reasoning problem on TKGs. With generated three datasets, we propose the first temporal CQE named Temporal Feature-Logic Embedding framework (TFLEX) to answer the temporal complex queries. We utilize vector logic to compute the logic part of Temporal Feature-Logic embeddings, thus naturally modeling all First-Order Logic (FOL) operations on entity set. In addition, our framework extends vector logic on timestamp set to cope with three extra temporal operators (After, Before and Between). Experiments on numerous query patterns demonstrate the effectiveness of our method.
... In what follows, we will introduce related work on both settings: TKG Reasoning under the interpolation setting. This setting aims to complete the missing facts at past timestamps (Jiang et al., 2016;Leblay and Chekol, 2018;Dasgupta et al., 2018;Garcia-Duran et al., 2018;Goel et al., 2020;Wu et al., 2020). For example, TTransE (Leblay and Chekol, 2018) extends TransE (Bordes et al., 2013) by adding the temporal constraints; HyTE (Dasgupta et al., 2018) projects the entities and relations to time-aware hyperplanes to generate representations for different timestamps. ...
... This setting aims to complete the missing facts at past timestamps (Jiang et al., 2016;Leblay and Chekol, 2018;Dasgupta et al., 2018;Garcia-Duran et al., 2018;Goel et al., 2020;Wu et al., 2020). For example, TTransE (Leblay and Chekol, 2018) extends TransE (Bordes et al., 2013) by adding the temporal constraints; HyTE (Dasgupta et al., 2018) projects the entities and relations to time-aware hyperplanes to generate representations for different timestamps. Above all, they cannot obtain the representations of the unseen timestamps and are not suitable for the extrapolation setting. ...
... Experimental Setup. We adopt three widelyused datasets, ICEWS14 (Li et al., 2021b), ICEWS18 (Jin et al., 2020), and WIKI (Leblay and Chekol, 2018) to evaluate CEN. Dataset statistics are demonstrated in Table 1. ...
Preprint
Full-text available
A Temporal Knowledge Graph (TKG) is a sequence of KGs corresponding to different timestamps. TKG reasoning aims to predict potential facts in the future given the historical KG sequences. One key of this task is to mine and understand evolutional patterns of facts from these sequences. The evolutional patterns are complex in two aspects, length-diversity and time-variability. Existing models for TKG reasoning focus on modeling fact sequences of a fixed length, which cannot discover complex evolutional patterns that vary in length. Furthermore, these models are all trained offline, which cannot well adapt to the changes of evolutional patterns from then on. Thus, we propose a new model, called Complex Evolutional Network (CEN), which uses a length-aware Convolutional Neural Network (CNN) to handle evolutional patterns of different lengths via an easy-to-difficult curriculum learning strategy. Besides, we propose to learn the model under the online setting so that it can adapt to the changes of evolutional patterns over time. Extensive experiments demonstrate that CEN obtains substantial performance improvement under both the traditional offline and the proposed online settings.
... The main challenge in TKGC is to additionally learn embeddings for time stamps, such that embedding models perform scoring jointly based on relation, entity and time stamp embeddings. This perspective has led to the development of several embedding models, building on static KGE models (Leblay and Chekol 2018;Lacroix, Obozinski, and Usunier 2020), or having dedicated neural architectures (Wu et al. 2020;García-Durán, Dumančić, and Niepert 2018), to appropriately represent temporal information. However, no current embedding model, to our knowledge, studies TKGC from the perspective of capturing temporal inference patterns despite their prevalence in real-world data (Toutanova and Chen 2015). ...
... Most TKGE models hence build on existing KGE models. For instance, TTransE (Leblay and Chekol 2018) extends TransE, and encodes time stamps as translations, analogously to relations, such that these translations additionally move head representations in the embedding space. ChronoR (Sadeghian et al. 2021) builds on RotatE, and represents time-relation pairs with rotation and scaling in the embedding space. ...
... Indeed, BoxTE achieves an MRR of 0.608 on ICEWS14, while the best ICEWS14 models ChronoR (Sadeghian et al. 2021) and TeLM (Xu et al. 2021) each report a value of less than 0.6. On ICEWS5-15, BoxTE achieves an MRR of 0.657 and thus surpasses TNTComplEx (Lacroix, Obozinski, and Usunier TBoxE implements the approach introduced by TTransE ( Leblay and Chekol 2018). In TTransE, each time stamp is represented as an additional relation embedding that acts on entity embeddings. ...
Preprint
Knowledge graph completion is the task of inferring missing facts based on existing data in a knowledge graph. Temporal knowledge graph completion (TKGC) is an extension of this task to temporal knowledge graphs, where each fact is additionally associated with a time stamp. Current approaches for TKGC primarily build on existing embedding models which are developed for (static) knowledge graph completion, and extend these models to incorporate time, where the idea is to learn latent representations for entities, relations, and timestamps and then use the learned representations to predict missing facts at various time steps. In this paper, we propose BoxTE, a box embedding model for TKGC, building on the static knowledge graph embedding model BoxE. We show that BoxTE is fully expressive, and possesses strong inductive capacity in the temporal setting. We then empirically evaluate our model and show that it achieves state-of-the-art results on several TKGC benchmarks.
... Depending on the type of knowledge, times are either particular time stamps or time intervals. In recent research, it has been shown that methods for static KGs are not always feasible for tKGs [5,7,12] and thus, new methods are needed for completion. ...
... The embedding models have been proven to have better performances on link prediction over tKGs, i.e, for answering queries in the form (source, relation, ?, timestamp), compared to traditional embedding models on static KGs. Nevertheless, some approaches are built upon former introduced heuristics on static KGs, like TTransE [12], TA-TransE [7], and HyTE [5], which use the translation-based method TransE [1] as their foundation. In the literature, we also have extensions of the well-known DistMult [21] model like TDistMult [14], the TA-DistMult proposed in [7] or Know-Evolve [18]. ...
... Hence, we store the transitions ( , , , +1 ) in D, and a batch is defined by a sample taken from the replay memory, ( , , , +1 ) ∼ D. Generally, it is beneficial to learn from the same transitions multiple times which greatly facilitates a stable learning process. [7] is a modification of FB15k [12] including the temporal information <occursSince> and <occursUn-til> for some facts. Because of its incompleteness regarding the temporal information, we adapt the concatenation step for calculating the fingerprints (c.f. ...
Preprint
Full-text available
In knowledge graph reasoning, we observe a trend to analyze temporal data evolving over time. The additional temporal dimension is attached to facts in a knowledge base resulting in quadruples between entities such as (Nintendo, released, Super Mario, Sep-13-1985), where the relation between two entities is associated to a specific time interval or point in time. Multi-hop reasoning on inferred subgraphs connecting entities within a knowledge graph can be formulated as a reinforcement learning task where the agent sequentially performs inference upon the explored subgraph. The task in this work is to infer the predicate between a subject and an object entity, i.e., (subject, ?, object, time), being valid at a certain timestamp or time interval. Given query entities, our agent starts to gather temporal relevant information about the neighborhood of the subject and object. The encoding of information about the explored graph structures is referred to as fingerprints. Subsequently, we use the two fingerprints as input to a Q-Network. Our agent decides sequentially which relational type needs to be explored next expanding the local subgraphs of the query entities in order to find promising paths between them. The evaluation shows that the proposed method not only yields results being in line with state-of-the-art embedding algorithms for temporal Knowledge Graphs (tKG), but we also gain information about the relevant structures between subjects and objects.
... The key idea is to encode the temporal fact into a low-dimensional embedding space, which allows complex vector operations [6]. These methods usually extend embedding methods designed for static knowledge graphs by incorporating the timestamp information into existing score functions, such as TTransE [7], TA-DistMult [8] and TNTComplEx [9]. Based on the time-dependent score functions, they can effectively leverage the timestamp information and measure the plausibility of temporal facts. ...
... These works usually compute a hidden representation for each timestamp and extend the static score functions to utilize temporal information. TTransE [7] and HyTE [24] can be seen as extensions of TransE [12], which consider the timestamp as the geometric translation and hyperplane in the embedding space. ...
... As described above, our method can be seen as an extension of existing TKG embedding models, which includes two modules, i.e., base model and relational constraints. Here, we select five widespread methods (TTransE [7], TA-DistMult [8], DE-DistMult [26], TNTComplEx [9], ChronoR [28]) as the base model to compute f (s, r, o, t) in (1). In terms of the relational constraints, we implement two variants of the constraints: RC td and RC hp , which correspond to tensor decomposition and hyperplane projection. ...
Article
Full-text available
Temporal knowledge graphs (TKGs) have become an effective tool for numerous intelligent applications. Due to their incompleteness, TKG embedding methods have been proposed to infer the missing temporal facts, and work by learning latent representations for entities, relations and timestamps. However, these methods primarily focus on measuring the plausibility of the whole temporal fact, and ignore the semantic property that there exists a bias between any relation and its involved entities at various time steps. In this paper, we present a novel temporal knowledge graph completion framework, which imposes relational constraints to preserve the semantic property implied in TKGs. Specifically, we borrow ideas from two well-known transformation functions, i.e., tensor decomposition and hyperplane projection, and design relational constraints associated with timestamps. We then adopt suitable regularization schemes to accommodate specific relational constraints, which combat overfitting and enforce temporal smoothness. Experimental studies indicate the superiority of our proposal compared to existing baselines on the task of temporal knowledge graph completion.
... The other is to embed temporal information on the basis of static knowledge graph. For example, TTransE [16] model adds vectorization of temporal information on the basis of TransE model and graphs it to the corresponding space for calculation. These methods have achieved good performance in completing temporal knowledge graph. ...
... These methods mainly add temporal information to calculate the similarity score. The most classical one is that TTransE [16] adds the projection of temporal information and carries out vector calculation on the basis of TransE, and modifies the distance calculation formula h + r ≈ t to s + r + t ≈ 0 to complete the temporal knowledge graph. Inspired by TransH, Jiang et al. put forward a new model called HyTE [11], which explicitly combines time in entity relation space by associating each timestamp with its corresponding hyperplane. ...
... This kind of model is mainly to embed the entity or the relation through modeling the relation structure. We adopt the following models for comparative experiments: TransE [2], DistMult [43], TTransE [16], TA-TransE [5] and TA-DistMult [5]. The parameter settings of all experimental datasets are exactly the same as the pre-processed few-shot datasets we used. ...
Article
Full-text available
Traditional knowledge graph completion mainly focuses on static knowledge graph. Although there are efforts studying temporal knowledge graph completion, they assume that each relation has enough entities to train, ignoring the influence of long tail relations. Moreover, many relations only have a few samples. In that case, how to handle few-shot temporal knowledge graph completion still merits further attention. This paper aims to propose a framework for completing few-shot temporal knowledge graph. We use self-attention mechanism to encode entities, use cyclic recursive aggregation network to aggregate reference sets, use fault-tolerant mechanism to deal with error information, and use similarity network to calculate similarity scores. Experimental results show that our proposed model outperforms the baseline models and has better stability.
... For example, they may confuse entities such as Barack Obama and Donald Trump when answering the query (?, is president of, US, 2019). Therefore, it is conceivable that incorporating time information during embedding learning can further improve the performance of KGE models and yield better embedding results [6,10,11,19,22,25]. ...
... Temporal KGE approaches More and more researchers find that incorporating time information during KG embedding learning can facilitate more effective results and multiple temporal KGE models have been proposed. Most of existing temporal KGE models can be seen as extensions of static models, such as, TTransE [22], TA-TransE [10], HyTE [6] and DE-TransE [11] are extended from TransE, TA-DistMult [10] and DE-DistMult [11] are temporal extensions of DistMult. Similar to their static versions TransE and DistMult, these temporal KGE models have issues with inferring symmetric relations or antisymmetric relations. ...
... r −1 is the inverse relation embedding of relation r [16]. For each t ∈ T , t ∈ R d and t ∈ C d are vectorial embeddings of the timestamp [19,22]. T ∈ R d 3 is the time-specific core tensor and × n denotes the tensor product along the n-th mode [25]. ...
Article
Full-text available
The approach of knowledge graph embedding (KGE) enables it possible to represent facts of a knowledge graph (KG) in low-dimensional continuous vector spaces. Consequently, it can significantly reduce the complexity of those operations performed on the underlying KG, and has attracted a lot of attention in recent years. However, most of KGE approaches have only been developed over static facts and ignore the time attribute. As a matter of effect, in some real-world KGs, a fact might only be valid for a specific time interval or point in time. For instance, the fact (Barack Obama, is president of, US, [2009-2017]) is only valid between 2009 and 2017. To conquer this issue, based on a famous tensor factorization approach, canonical polyadic decomposition, we propose two new temporal KGE models called TSimplE and TNTSimplE that integrates time information besides static facts. A non-temporal component is also added to deal with heterogeneous temporal KGs that include both temporal and non-temporal relations. We prove that the proposed models are fully expressive which has a bound on the dimensionality of their embeddings, and can incorporate several important types of background knowledge including symmetry, antisymmetry and inversion. In addition, our models are capable of dealing with two common challenges in real-world temporal KGs, i.e., modeling time intervals and predicting time for facts with missing time information. We conduct extensive experiments on three real-world temporal KGs: ICEWS, YAGO3 and Wikidata. The results indicate that our models achieve start-of-the-art performance with lower time or space complexity.
... They mainly address two problem setups, i.e., interpolation and extrapolation. Given a TKG ranging from time 0 to time , methods for the interpolation setup [4,8,21] infer missing facts for time (0 ≤ ≤ ); on the other hand, those for the extrapolation setup [17,41,42] predict new facts for time > . In this paper, we focus on the extrapolation setting, which is more challenging and interesting than the other setting, as forecasting emerging events are of great importance to many applications of TKG reasoning. ...
... We use five real-world TKGs that have been widely used in previous studies: ICEWS18 [2], ICEWS14 [41], GDELT [22], WIKI [21], and YAGO [23]. ICEWS (Integrated Crisis Early Warning System) and GDELT (Global Database of Events, Language, and Tone) are eventbased TKGs; WIKI and YAGO are knowledge bases with temporally associated facts. ...
... Reasoning over Dynamic Heterogeneous Graphs. Static KG embedding methods have been extended to take temporal information into account, including TA-DistMult [8], TTransE [21], HyTE [4], and diachronic embedding [9]. These temporal KG embedding techniques address an interpolation problem where the goal is to infer missing facts at some point in the past, and cannot predict future events. ...
Preprint
How can we perform knowledge reasoning over temporal knowledge graphs (TKGs)? TKGs represent facts about entities and their relations, where each fact is associated with a timestamp. Reasoning over TKGs, i.e., inferring new facts from time-evolving KGs, is crucial for many applications to provide intelligent services. However, despite the prevalence of real-world data that can be represented as TKGs, most methods focus on reasoning over static knowledge graphs, or cannot predict future events. In this paper, we present a problem formulation that unifies the two major problems that need to be addressed for an effective reasoning over TKGs, namely, modeling the event time and the evolving network structure. Our proposed method EvoKG jointly models both tasks in an effective framework, which captures the ever-changing structural and temporal dynamics in TKGs via recurrent event modeling, and models the interactions between entities based on the temporal neighborhood aggregation framework. Further, EvoKG achieves an accurate modeling of event time, using flexible and efficient mechanisms based on neural density estimation. Experiments show that EvoKG outperforms existing methods in terms of effectiveness (up to 77% and 116% more accurate time and link prediction) and efficiency.
... Several embedding-based methods have been introduced for tKGs to solve link prediction and forecasting (link prediction with future timestamps), e.g., TTransE (Leblay and Chekol 2018), TNTComplEx (Lacroix, Obozinski, and Usunier 2020), and RE-Net (Jin et al. 2019). The underlying principle is to project the entities and relations into a lowdimensional vector space while preserving the topology and temporal dynamics of the tKG. ...
... Well-known methods include RESCAL (Nickel, Tresp, andKriegel 2011), TransE (Bordes et al. 2013), DistMult (Yang et al. 2015), and Com-plEx (Trouillon et al. 2016) as well as the graph convolutional approaches R-GCN (Schlichtkrull et al. 2018) and CompGCN (Vashishth et al. 2020). Several approaches have been recently proposed to handle tKGs, such as TTransE (Leblay and Chekol 2018), TA-DistMult (García-Durán, Dumancić, and Niepert 2018), DE-SimplE (Goel et al. 2020), TNTComplEx (Lacroix, Obozinski, and Usunier 2020), CyGNet (Zhu et al. 2021), RE-Net (Jin et al. 2019), and xERTE (Han et al. 2021). The main idea of these methods is to explicitly learn embeddings for timestamps or to integrate temporal information into the entity or relation embeddings. ...
... Baseline methods We compare TLogic 3 with the stateof-the-art baselines for static link prediction DistMult (Yang et al. 2015), ComplEx (Trouillon et al. 2016), and Any-BURL (Meilicke et al. 2019(Meilicke et al. , 2020 as well as for temporal link prediction TTransE (Leblay and Chekol 2018), TA-DistMult (García-Durán, Dumancić, and Niepert 2018), DE-SimplE (Goel et al. 2020), TNTComplEx (Lacroix, Obozinski, and Usunier 2020), CyGNet (Zhu et al. 2021), RE-Net (Jin et al. 2019), and xERTE (Han et al. 2021). All baseline results except for the results on AnyBURL are from Han et al. (2021). ...
Article
Conventional static knowledge graphs model entities in relational data as nodes, connected by edges of specific relation types. However, information and knowledge evolve continuously, and temporal dynamics emerge, which are expected to influence future situations. In temporal knowledge graphs, time information is integrated into the graph by equipping each edge with a timestamp or a time range. Embedding-based methods have been introduced for link prediction on temporal knowledge graphs, but they mostly lack explainability and comprehensible reasoning chains. Particularly, they are usually not designed to deal with link forecasting -- event prediction involving future timestamps. We address the task of link forecasting on temporal knowledge graphs and introduce TLogic, an explainable framework that is based on temporal logical rules extracted via temporal random walks. We compare TLogic with state-of-the-art baselines on three benchmark datasets and show better overall performance while our method also provides explanations that preserve time consistency. Furthermore, in contrast to most state-of-the-art embedding-based methods, TLogic works well in the inductive setting where already learned rules are transferred to related datasets with a common vocabulary.
... Temporal KG reasoning As discussed by Jin et al. [2020], temporal KG reasoning can be divided into two task settings that aim at predicting facts that are positioned differently on the timeline. In the interpolation setting, the models [Jiang et al., 2016, Sadeghian et al., 2016, Dasgupta et al., 2018, García-Durán et al., 2018, Leblay and Chekol, 2018, Goel et al., 2020, Montella et al., 2021 infer missing facts at the historical timestamps. To do so, Dasgupta et al. [2018] projected the entities and relations onto timestamp-specific hyperplanes. ...
... To do so, Dasgupta et al. [2018] projected the entities and relations onto timestamp-specific hyperplanes. Leblay and Chekol [2018] and García-Durán et al. [2018] considered the time as a second relation and integrated times with relations. ...
... Experimental setup We use four representative TKGs datasets, GDELT [Leetaru and Schrodt, 2013], ICEWS18 [Boschee et al., 2015], WIKI [Leblay and Chekol, 2018], and YAGO [Mahdisoltani et al., 2014]. More details about the datasets can be found in Table 1 and appendix A.3. ...
Preprint
Recently there is an increasing scholarly interest in time-varying knowledge graphs, or temporal knowledge graphs (TKG). Previous research suggests diverse approaches to TKG reasoning that uses historical information. However, less attention has been given to the hierarchies within such information at different timestamps. Given that TKG is a sequence of knowledge graphs based on time, the chronology in the sequence derives hierarchies between the graphs. Furthermore, each knowledge graph has its hierarchical level which may differ from one another. To address these hierarchical characteristics in TKG, we propose HyperVC, which utilizes hyperbolic space that better encodes the hierarchies than Euclidean space. The chronological hierarchies between knowledge graphs at different timestamps are represented by embedding the knowledge graphs as vectors in a common hyperbolic space. Additionally, diverse hierarchical levels of knowledge graphs are represented by adjusting the curvatures of hyperbolic embeddings of their entities and relations. Experiments on four benchmark datasets show substantial improvements, especially on the datasets with higher hierarchical levels.
... To address the inherent incompleteness of temporal KGs, Tresp et al. (2015) proposed the first tKG model. Afterwards, a line of work emerged that extends static KG completion models by adding temporal embeddings, e.g., TTransE (Leblay and Chekol, 2018), TA-TransE (García-Durán et al., 2018), DE-SimplE (Goel et al., 2019), TNTCom-plEx (Lacroix et al., 2020), ConT (Ma et al., 2018), and many more. The models generally consist of two parts, a temporal embedding layer to capture the evolving features of tKGs and a score function to examine the plausibility of a given quadruple. ...
... Temporal embeddings are crucial in temporal KG completion models for storing the evolving knowledge; without them, the temporal aspect cannot be captured. The PEs can be generally categorized into three classes: (1) timestamp embeddings (TEs): the models learn an embedding for each discrete timestamp in the same vector space as entities and relations (Tresp et al., 2017;Leblay and Chekol, 2018;Dasgupta et al., 2018;Lacroix et al., 2020). (2) time-dependent entity embeddings (TEEs): the models define entity embedding as a function that takes an entity and a timestamp as input and generates a time-dependent representation for the entity at that time (Goel et al., 2019;Xu et al., 2019;Han et al., 2020a). ...
... Regarding the first aim, we surprisingly find that the TE proposed by Leblay and Chekol (2018) outperforms other temporal embedding approaches on the ICEWS subsets and achieves on-par results on GDELT. Leblay and Chekol (2018) represent timestamps in the same vector space as entities and relations and learn embeddings for each discrete timestamp. ...
... In what follows, we will introduce related work on both settings: TKG Reasoning under the interpolation setting. This setting aims to complete the missing facts at past timestamps (Jiang et al., 2016;Leblay and Chekol, 2018;Dasgupta et al., 2018;Garcia-Duran et al., 2018;Goel et al., 2020;Wu et al., 2020). For example, TTransE (Leblay and Chekol, 2018) extends TransE (Bordes et al., 2013) by adding the temporal constraints; HyTE (Dasgupta et al., 2018) projects the entities and relations to time-aware hyperplanes to generate representations for different timestamps. ...
... This setting aims to complete the missing facts at past timestamps (Jiang et al., 2016;Leblay and Chekol, 2018;Dasgupta et al., 2018;Garcia-Duran et al., 2018;Goel et al., 2020;Wu et al., 2020). For example, TTransE (Leblay and Chekol, 2018) extends TransE (Bordes et al., 2013) by adding the temporal constraints; HyTE (Dasgupta et al., 2018) projects the entities and relations to time-aware hyperplanes to generate representations for different timestamps. Above all, they cannot obtain the representations of the unseen timestamps and are not suitable for the extrapolation setting. ...
... Experimental Setup. We adopt three widelyused datasets, ICEWS14 (Li et al., 2021b), ICEWS18 (Jin et al., 2020), and WIKI (Leblay and Chekol, 2018) CyGNet (Zhu et al., 2021), RE-NET (Jin et al., 2020), xERTE (Han et al., 2020a), TG-Tucker , TG-DistMult , TiTer (Sun et al., 2021) and RE-GCN (Li et al., 2021b). In the experiments, we adopt MRR (Mean Reciprocal Rank) and Hits@{1,3,10} as the metrics for TKG reasoning. ...
... Temporal KGC Many TKGC models incorporate additional time-specific parameters upon existing KGC methods. Leblay and Chekol (2018), based on Bordes et al. (2013), represents each timestamp with independent embeddings. Dasgupta et al. (2018) resembles Wang et al. (2014), regarding timestamps as hyperplanes for entities to project. ...
... TTransE (Leblay and Chekol, 2018 by the low quality of entities in ICEWS14, which only includes the "sector" and "country" of the entities. These entity descriptions are much less informative than the ones in the SKGC benchmark. ...
Preprint
Full-text available
Knowledge Graph Completion (KGC) has been recently extended to multiple knowledge graph (KG) structures, initiating new research directions, e.g. static KGC, temporal KGC and few-shot KGC. Previous works often design KGC models closely coupled with specific graph structures, which inevitably results in two drawbacks: 1) structure-specific KGC models are mutually incompatible; 2) existing KGC methods are not adaptable to emerging KGs. In this paper, we propose KG-S2S, a Seq2Seq generative framework that could tackle different verbalizable graph structures by unifying the representation of KG facts into "flat" text, regardless of their original form. To remedy the KG structure information loss from the "flat" text, we further improve the input representations of entities and relations, and the inference algorithm in KG-S2S. Experiments on five benchmarks show that KG-S2S outperforms many competitive baselines, setting new state-of-the-art performance. Finally, we analyze KG-S2S's ability on the different relations and the Non-entity Generations.
... With the development of TKGs, TKG embedding (TKGE) draws increasing attention (Leblay and Chekol, 2018;Xu et al., 2020cXu et al., ,a, 2021Lacroix et al., 2020). An example of typical TKGE models is TTransE (Leblay and Chekol, 2018) which represents timestamps as latent vectors with entities and relations and incorporates time embeddings into its score function ||e s + r + τ − e o ||. ...
... With the development of TKGs, TKG embedding (TKGE) draws increasing attention (Leblay and Chekol, 2018;Xu et al., 2020cXu et al., ,a, 2021Lacroix et al., 2020). An example of typical TKGE models is TTransE (Leblay and Chekol, 2018) which represents timestamps as latent vectors with entities and relations and incorporates time embeddings into its score function ||e s + r + τ − e o ||. The success of TKGE models shows that the inclusion of time information is helpful for reasoning over TKGs. ...
Preprint
Full-text available
Entity alignment aims to identify equivalent entity pairs between different knowledge graphs (KGs). Recently, the availability of temporal KGs (TKGs) that contain time information created the need for reasoning over time in such TKGs. Existing embedding-based entity alignment approaches disregard time information that commonly exists in many large-scale KGs, leaving much room for improvement. In this paper, we focus on the task of aligning entity pairs between TKGs and propose a novel Time-aware Entity Alignment approach based on Graph Neural Networks (TEA-GNN). We embed entities, relations and timestamps of different KGs into a vector space and use GNNs to learn entity representations. To incorporate both relation and time information into the GNN structure of our model, we use a time-aware attention mechanism which assigns different weights to different nodes with orthogonal transformation matrices computed from embeddings of the relevant relations and timestamps in a neighborhood. Experimental results on multiple real-world TKG datasets show that our method significantly outperforms the state-of-the-art methods due to the inclusion of time information.
... Leblay et al. [31] extended the traditional TransE into a temporal embedding model TTransE via an additional embedding transformation: from timestamps to hidden representations. For each temporal fact (s, p, o, t), time t is also embedded in the same feature space as entities and relations. ...
... • TTransE [31] represents entities, relations and timestamps in a uniform low-dimensional feature space, and regards relations as translation calculations to concatenate the entities and timestamps. • TA-DistMult [33] leverages LSTM networks to transform temporal information to time-aware representations, then the embedding vectors process relations to form time-aware relation embeddings for score function. ...
Preprint
Research on knowledge graph embedding (KGE) has emerged as an active field in which most existing KGE approaches mainly focus on static structural data and ignore the influence of temporal variation involved in time-aware triples. In order to deal with this issue, several temporal knowledge graph embedding (TKGE) approaches have been proposed to integrate temporal and structural information in recent years. However, these methods only employ a uniformly random sampling to construct negative facts. As a consequence, the corrupted samples are often too simplistic for training an effective model. In this paper, we propose a new temporal knowledge graph embedding framework by introducing adversarial learning to further refine the performance of traditional TKGE models. In our framework, a generator is utilized to construct high-quality plausible quadruples and a discriminator learns to obtain the embeddings of entities and relations based on both positive and negative samples. Meanwhile, we also apply a Gumbel-Softmax relaxation and the Wasserstein distance to prevent vanishing gradient problems on discrete data; an inherent flaw in traditional generative adversarial networks. Through comprehensive experimentation on temporal datasets, the results indicate that our proposed framework can attain significant improvements based on benchmark models and also demonstrate the effectiveness and applicability of our framework.
... Several embedding-based methods have been introduced for tKGs to solve link prediction and forecasting (link prediction with future timestamps), e.g., TTransE (Leblay and Chekol 2018), TNTComplEx (Lacroix, Obozinski, and Usunier 2020), and RE-Net (Jin et al. 2019). The underlying principle is to project the entities and relations into a lowdimensional vector space while preserving the topology and temporal dynamics of the tKG. ...
... Well-known methods include RESCAL (Nickel, Tresp, andKriegel 2011), TransE (Bordes et al. 2013), DistMult (Yang et al. 2015), and Com-plEx (Trouillon et al. 2016) as well as the graph convolutional approaches R- GCN (Schlichtkrull et al. 2018) and CompGCN (Vashishth et al. 2020). Several approaches have been recently proposed to handle tKGs, such as TTransE (Leblay and Chekol 2018), TA-DistMult (García-Durán, Dumancić, and Niepert 2018), DE-SimplE (Goel et al. 2020), TNTComplEx (Lacroix, Obozinski, and Usunier 2020), CyGNet (Zhu et al. 2021), RE-Net (Jin et al. 2019), and xERTE (Han et al. 2021). The main idea of these methods is to explicitly learn embeddings for timestamps or to integrate temporal information into the entity or relation embeddings. ...
Preprint
Conventional static knowledge graphs model entities in relational data as nodes, connected by edges of specific relation types. However, information and knowledge evolve continuously, and temporal dynamics emerge, which are expected to influence future situations. In temporal knowledge graphs, time information is integrated into the graph by equipping each edge with a timestamp or a time range. Embedding-based methods have been introduced for link prediction on temporal knowledge graphs, but they mostly lack explainability and comprehensible reasoning chains. Particularly, they are usually not designed to deal with link forecasting -- event prediction involving future timestamps. We address the task of link forecasting on temporal knowledge graphs and introduce TLogic, an explainable framework that is based on temporal logical rules extracted via temporal random walks. We compare TLogic with state-of-the-art baselines on three benchmark datasets and show better overall performance while our method also provides explanations that preserve time consistency. Furthermore, in contrast to most state-of-the-art embedding-based methods, TLogic works well in the inductive setting where already learned rules are transferred to related datasets with a common vocabulary.
... With the development of TKGs, TKG embedding (TKGE) draws increasing attention (Leblay and Chekol, 2018;Xu et al., 2020cXu et al., ,a, 2021Lacroix et al., 2020). An example of typical TKGE models is TTransE (Leblay and Chekol, 2018) which represents timestamps as latent vectors with entities and relations and incorporates time embeddings into its score function ||e s + r + τ − e o ||. ...
... With the development of TKGs, TKG embedding (TKGE) draws increasing attention (Leblay and Chekol, 2018;Xu et al., 2020cXu et al., ,a, 2021Lacroix et al., 2020). An example of typical TKGE models is TTransE (Leblay and Chekol, 2018) which represents timestamps as latent vectors with entities and relations and incorporates time embeddings into its score function ||e s + r + τ − e o ||. The success of TKGE models shows that the inclusion of time information is helpful for reasoning over TKGs. ...
... Knowledge Graphs (KGs) collect and store human 21 knowledge as large multi-relational directed graphs 22 where nodes represent entities, and typed edges rep- 23 resent relationships between entities. Examples of 24 real-world KGs include Freebase [1], YAGO [2] and 25 WordNet [3]. ...
... • is the element-wise product of vectors[15].For each τ ∈ T , τ ∈ R d and τ ∈ C d are vectorial embeddings of timestamps[22,23]. x ⊥ = x − τ T xτwhere x ∈ {h, r, t} denotes the temporal projection relation as a translation from the head to the tail.The score function is defined as the Euclidean dis- ...
Article
Full-text available
Knowledge Graph (KG) embedding approaches have been proved effective to infer new facts for a KG based on the existing ones–a problem known as KG completion. However, most of them have focused on static KGs, in fact, relational facts in KGs often show temporal dynamics, e.g., the fact (US, has president, Barack Obama, [2009–2017]) is only valid from 2009 to 2017. Therefore, utilizing available time information to develop temporal KG embedding models is an increasingly important problem. In this paper, we propose a new hyperplane-based time-aware KG embedding model for temporal KG completion. By employing the method of time-specific hyperplanes, our model could explicitly incorporate time information in the entity-relation space to predict missing elements in the KG more effectively, especially temporal scopes for facts with missing time information. Moreover, in order to model and infer four important relation patterns including symmetry, antisymmetry, inversion and composition, we map facts happened at the same time into a polar coordinate system. During training procedure, a time-enhanced negative sampling strategy is proposed to get more effective negative samples. Experimental results on datasets extracted from real-world temporal KGs show that our model significantly outperforms existing state-of-the-art approaches for the KG completion task.
... For static baselines, we use TransE (Bordes et al., 2013), DistMult , RotatE (Sun et al., 2019), and QuatE (Zhang et al., 2019a). For TKGE methods, we consider TTransE (Leblay and Chekol, 2018) (Xu et al., 2020a). 7 Note that TeRo (Xu et al., 2020a) is also based on the idea of rotations, and thus we consider TeRo as a directly baseline. ...
... Many aforementioned methods (Dasgupta et al., 2018;Leblay and Chekol, 2018;Trivedi et al., 2017;García-Durán et al., 2018;Goel et al., 2020;Sadeghian et al., 2021) are extended from static Static KGs to TKGs. They integrate time information into previous static methods as independent features. ...
Preprint
Full-text available
Temporal factors are tied to the growth of facts in realistic applications, such as the progress of diseases and the development of political situation, therefore, research on Temporal Knowledge Graph (TKG) attracks much attention. In TKG, relation patterns inherent with temporality are required to be studied for representation learning and reasoning across temporal facts. However, existing methods can hardly model temporal relation patterns, nor can capture the intrinsic connections between relations when evolving over time, lacking of interpretability. In this paper, we propose a novel temporal modeling method which represents temporal entities as Rotations in Quaternion Vector Space (RotateQVS) and relations as complex vectors in Hamilton's quaternion space. We demonstrate our method can model key patterns of relations in TKG, such as symmetry, asymmetry, inverse, and can further capture time-evolved relations by theory. Empirically, we show that our method can boost the performance of link prediction tasks over four temporal knowledge graph benchmarks.
... Datasets. We use four public TKG datasets for evaluation: ICEWS14, ICEWS18 (Boschee et al., 2015), WIKI (Leblay and Chekol, 2018a), and YAGO (Mahdisoltani et al., 2015). Integrated Crisis Early Warning System (ICEWS) is an event dataset. ...
... Event argument completion is then conducted based on the time-aware score learner upon the time-specific embeddings of event elements in the graph sequence. García-Durán et al. [157] and Leblay and Chekol [158] integrated the time when the events occurred into the embeddings of event types by concatenating their embeddings. Then, the score learner was designed on these time-aware embeddings of event types and the embeddings of other arguments as their combination operation like TransE [159]. ...
Preprint
Full-text available
Besides entity-centric knowledge, usually organized as Knowledge Graph (KG), events are also an essential kind of knowledge in the world, which trigger the spring up of event-centric knowledge representation form like Event KG (EKG). It plays an increasingly important role in many machine learning and artificial intelligence applications, such as intelligent search, question-answering, recommendation, and text generation. This paper provides a comprehensive survey of EKG from history, ontology, instance, and application views. Specifically, to characterize EKG thoroughly, we focus on its history, definitions, schema induction, acquisition, related representative graphs/systems, and applications. The development processes and trends are studied therein. We further summarize perspective directions to facilitate future research on EKG.
... TTransE, TA-TransE (Leblay and Chekol 2018): They are based on the TransE model and introduce a time processing strategy to adapt to the reasoning task of a temporal knowledge graph. ...
Article
The time series data in the manufacturing process reflects the sequential state of the manufacturing system, and the fusion of temporal features into the industrial knowledge graph will undoubtedly significantly improve the knowledge process efficiency of the manufacturing system. This paper proposes a semantic-aware event link reasoning over an industrial knowledge graph embedding time series data. Its knowledge graph skeleton is constructed through a specific manufacturing process. NLTK is used to transform technical documents into a structured industrial knowledge graph. We employ deep learning (DL)-based models to obtain semantic information related to product quality prediction using time series data collected from IoT devices. Then the prediction information is attached to the specified node in the knowledge graph. Thus, the knowledge graph will describe the dynamic semantic information of manufacturing contexts. Meanwhile, a dynamic event link reasoning model that uses graph embedding to aggregate manufacturing processes information is proposed. The implicit information with industrial temporal knowledge can be further mined and inferred. The case study has shown that the proposed knowledge graph link reasoning reflects dynamic temporal characteristics. Compared to the classical knowledge graph prediction models, our model is superior to the baseline methods.
... Recent works follow a common paradigm, that is, to encode time as embeddings and then incorporate them into time-agnostic KBC models [7-9, 11, 12, 14]. Leblay and Chekol [12] investigated several extensions of existing KBC models by directly fusing time embeddings with relation embeddings, including TTransE, TRESCAL, etc. Goel et al. [8] proposed to learn time-varying entity embeddings by replacing a fraction of embedding weights with an activation function of learned frequencies. Unlike previous work, which view time as numeric values, García-Durán et al. [7] concatenated the string representation of the relation and the time, and fed them into an LSTM to obtain time-aware relation representations, which were used in TransE (TA-TransE) and DistMult(TA-DM) afterwards. ...
Preprint
Full-text available
Almost all statements in knowledge bases have a temporal scope during which they are valid. Hence, knowledge base completion (KBC) on temporal knowledge bases (TKB), where each statement \textit{may} be associated with a temporal scope, has attracted growing attention. Prior works assume that each statement in a TKB \textit{must} be associated with a temporal scope. This ignores the fact that the scoping information is commonly missing in a KB. Thus prior work is typically incapable of handling generic use cases where a TKB is composed of temporal statements with/without a known temporal scope. In order to address this issue, we establish a new knowledge base embedding framework, called TIME2BOX, that can deal with atemporal and temporal statements of different types simultaneously. Our main insight is that answers to a temporal query always belong to a subset of answers to a time-agnostic counterpart. Put differently, time is a filter that helps pick out answers to be correct during certain periods. We introduce boxes to represent a set of answer entities to a time-agnostic query. The filtering functionality of time is modeled by intersections over these boxes. In addition, we generalize current evaluation protocols on time interval prediction. We describe experiments on two datasets and show that the proposed method outperforms state-of-the-art (SOTA) methods on both link prediction and time prediction.
... Of late, understanding large KGs as a dynamic body of knowledge has gained attention, giving rise to the notion of temporal knowledge graphs or temporal knowledge bases [25,70]. Here, each edge (corresponding to a fact) is associated with a temporal scope or validity [43], with current efforts mostly focusing on the topic of temporal KG completion [31,32,42]. A very recent approach has explored QA over such temporal KGs, along with the creation of an associated benchmark [57]. ...
Preprint
Full-text available
Question answering over knowledge graphs (KG-QA) is a vital topic in IR. Questions with temporal intent are a special class of practical importance, but have not received much attention in research. This work presents EXAQT, the first end-to-end system for answering complex temporal questions that have multiple entities and predicates, and associated temporal conditions. EXAQT answers natural language questions over KGs in two stages, one geared towards high recall, the other towards precision at top ranks. The first step computes question-relevant compact subgraphs within the KG, and judiciously enhances them with pertinent temporal facts, using Group Steiner Trees and fine-tuned BERT models. The second step constructs relational graph convolutional networks (R-GCNs) from the first step's output, and enhances the R-GCNs with time-aware entity embeddings and attention over temporal relations. We evaluate EXAQT on TimeQuestions, a large dataset of 16k temporal questions we compiled from a variety of general purpose KG-QA benchmarks. Results show that EXAQT outperforms three state-of-the-art systems for answering complex questions over KGs, thereby justifying specialized treatment of temporal QA.
... We evaluated the performance of SeDyT on five popular datasets: GDELT [15], ICEWS14 [25], ICEWS18 [4], WIKI [14], and YAGO [16]. We follow the same training, validation, and test split used in the event forecasting works [9,11,33]. ...
Preprint
Temporal Knowledge Graphs store events in the form of subjects, relations, objects, and timestamps which are often represented by dynamic heterogeneous graphs. Event forecasting is a critical and challenging task in Temporal Knowledge Graph reasoning that predicts the subject or object of an event in the future. To obtain temporal embeddings multi-step away in the future, existing methods learn generative models that capture the joint distribution of the observed events. To reduce the high computation costs, these methods rely on unrealistic assumptions of independence and approximations in training and inference. In this work, we propose SeDyT, a discriminative framework that performs sequence modeling on the dynamic entity embeddings to solve the multi-step event forecasting problem. SeDyT consists of two components: a Temporal Graph Neural Network that generates dynamic entity embeddings in the past and a sequence model that predicts the entity embeddings in the future. Compared with the generative models, SeDyT does not rely on any heuristic-based probability model and has low computation complexity in both training and inference. SeDyT is compatible with most Temporal Graph Neural Networks and sequence models. We also design an efficient training method that trains the two components in one gradient descent propagation. We evaluate the performance of SeDyT on five popular datasets. By combining temporal Graph Neural Network models and sequence models, SeDyT achieves an average of 2.4% MRR improvement when not using the validation set and more than 10% MRR improvement when using the validation set.
... From the static KG embedding models, we use TransE [Bordes et al., 2013], DistMult [Yang et al., 2014], ComplEx [Trouillon et al., 2016], and SimplE [Kazemi and Poole, 2018] where these methods ignore the available time information. From the temporal KG embedding models, we compare the performance of our model with several state-of-the-art methods, including TTransE [Leblay and Chekol, 2018], AiTSEE , DE-Simple [Goel et al., 2020], TNTComplE [Lacroix et al., 2020], DyERNIE 6 [Han et al., 2020b], and TeRO [Xu et al., 2020]. We also report results of other ECOLA variants introduced in Section 3.6. ...
Preprint
Full-text available
With the emerging research effort to integrate structured and unstructured knowledge, many approaches incorporate factual knowledge into pre-trained language models (PLMs) and apply the knowledge-enhanced PLMs on downstream NLP tasks. However, (1) they only consider \textit{static} factual knowledge, but knowledge graphs (KGs) also contain \textit{temporal facts} or \textit{events} indicating evolutionary relationships among entities at different timestamps. (2) PLMs cannot be directly applied to many KG tasks, such as temporal KG completion. In this paper, we focus on \textbf{e}nhancing temporal knowledge embeddings with \textbf{co}ntextualized \textbf{la}nguage representations (ECOLA). We align structured knowledge contained in temporal knowledge graphs with their textual descriptions extracted from news articles and propose a novel knowledge-text prediction task to inject the abundant information from descriptions into temporal knowledge embeddings. ECOLA jointly optimizes the knowledge-text prediction objective and the temporal knowledge embeddings, which can simultaneously take full advantage of textual and knowledge information. For training ECOLA, we introduce three temporal KG datasets with aligned textual descriptions. Experimental results on the temporal knowledge graph completion task show that ECOLA outperforms state-of-the-art temporal KG models by a large margin. The proposed datasets can serve as new temporal KG benchmarks and facilitate future research on structured and unstructured knowledge integration.
... Datasets. We use four public TKG datasets for evaluation: ICEWS14, ICEWS18 (Boschee et al., 2015), WIKI (Leblay and Chekol, 2018a), and YAGO (Mahdisoltani et al., 2015). Integrated Crisis Early Warning System (ICEWS) is an event dataset. ...
Preprint
Temporal knowledge graph (TKG) reasoning is a crucial task that has gained increasing research interest in recent years. Most existing methods focus on reasoning at past timestamps to complete the missing facts, and there are only a few works of reasoning on known TKGs to forecast future facts. Compared with the completion task, the forecasting task is more difficult that faces two main challenges: (1) how to effectively model the time information to handle future timestamps? (2) how to make inductive inference to handle previously unseen entities that emerge over time? To address these challenges, we propose the first reinforcement learning method for forecasting. Specifically, the agent travels on historical knowledge graph snapshots to search for the answer. Our method defines a relative time encoding function to capture the timespan information, and we design a novel time-shaped reward based on Dirichlet distribution to guide the model learning. Furthermore, we propose a novel representation method for unseen entities to improve the inductive inference ability of the model. We evaluate our method for this link prediction task at future timestamps. Extensive experiments on four benchmark datasets demonstrate substantial performance improvement meanwhile with higher explainability, less calculation, and fewer parameters when compared with existing state-of-the-art methods.
... For example, HyTE [8] uses time representations to project knowledge at each snapshot to a timespecific hyperplane and then applies TransE in each hyperplane to learn the representations for each snapshot. TTransE [32] first adds a relation representation and time representation to obtain a translation vector and then uses it to translate the subject entity to the object entity in a vector space. TNTComplEX [9] regards a temporal knowledge graph as a 4-order tensor and learns the TKG representation via a tensor decomposition based on a new temporal regularization scheme. ...
Article
Full-text available
Temporal knowledge graph (TKG) representation learning aims to project entities and relations in TKGs to a low-dimensional vector space while preserving the evolutionary nature of TKGs. Most existing methods treat knowledge that happens at different times separately, which fails to explore how temporal knowledge graphs evolve over time. Actually, TKGs should evolve both on the local and global structures. The local structure evolution describes the formation process of the graph structure in a detailed manner, while the global structure evolution refers to the dynamic topology (e.g., community partition) of the graph, which is derived from the continuous formation process. Both are key factors for understanding the evolutionary nature of the TKGs. Unfortunately, little attention has been given to this area of research. In this paper, we propose a new TKG representation learning framework with local and global structure evolutions, named EvoExplore. Specifically, we define the local structure evolution as an establishment process of the relations between the entities, and propose a hierarchical-attention-based temporal point process to capture the formation process of the graph structure in a fine-grained manner. For global structure evolution, we propose a novel soft modularity parameterized by the entity representations to capture the dynamic community partition of the TKGs. Finally, we employ a multi-task loss function to jointly optimize the above two parts, which allows EvoExplore to learn the mutual influences of the local and global structure evolutions. Experimental results on three real-world datasets demonstrate the superiority of EvoExplore compared with the baseline methods. Code is available at https://github.com/zjs123/EvoExplore and https://github.com/zjs123/EvoExplore_MindSpore.
... Time features are considered individual input signals, favoring a more expressive inclusion of time information within the model. A prominent example for inclusion-based time extension is TTransE (Leblay and Chekol, 2018), representing time as temporal translation of entityrelation features. Similarly, TeRo (Xu et al., 2020a) considers individual time features via temporal rotation of entity features. ...
Preprint
Full-text available
Temporal knowledge graph completion (TKGC) has become a popular approach for reasoning over the event and temporal knowledge graphs, targeting the completion of knowledge with accurate but missing information. In this context, tensor decomposition has successfully modeled interactions between entities and relations. Their effectiveness in static knowledge graph completion motivates us to introduce Time-LowFER, a family of parameter-efficient and time-aware extensions of the low-rank tensor factorization model LowFER. Noting several limitations in current approaches to represent time, we propose a cycle-aware time-encoding scheme for time features, which is model-agnostic and offers a more generalized representation of time. We implement our methods in a unified temporal knowledge graph embedding framework, focusing on time-sensitive data processing. The experiments show that our proposed methods perform on par or better than the state-of-the-art semantic matching models on two benchmarks.
... We use the UCI (Panzarasa, Opsahl, and Carley 2009), Enron (Klimt and Yang 2004), Yelp 3 , ML-10M (Harper and Konstan 2015), WIKI (Leblay and Chekol 2018), and YAGO (Mahdisoltani, Biega, and Suchanek 2014) datasets in our experiments. Table 1 summarizes the datasets and additional information is provided in the supplementary material. ...
Article
Although static networks have been extensively studied in machine learning, data mining, and AI communities for many decades, the study of dynamic networks has recently taken center stage due to the prominence of social media and its effects on the dynamics of social networks. In this paper, we propose a statistical model for dynamically evolving networks, together with a variational inference approach. Our model, Neural Latent Space Model with Variational Inference, encodes edge dependencies across different time snapshots. It represents nodes via latent vectors and uses interaction matrices to model the presence of edges. These matrices can be used to incorporate multiple relations in heterogeneous networks by having a separate matrix for each of the relations. To capture the temporal dynamics, both node vectors and interaction matrices are allowed to evolve with time. Existing network analysis methods use representation learning techniques for modelling networks. These techniques are different for homogeneous and heterogeneous networks because heterogeneous networks can have multiple types of edges and nodes as opposed to a homogeneous network. Unlike these, we propose a unified model for homogeneous and heterogeneous networks in a variational inference framework. Moreover, the learned node latent vectors and interaction matrices may be interpretable and therefore provide insights on the mechanisms behind network evolution. We experimented with a single step and multi-step link forecasting on real-world networks of homogeneous, bipartite, and heterogeneous nature, and demonstrated that our model significantly outperforms existing models.
... While many of the previous works study knowledge graph completion on static KGs, little attention has been given to temporally-aware KGs. Though recent work has begun to solve the temporal link prediction task, these models often utilize a large number of parameters, making them difficult to train (Garcia-Duran, Dumančić, and Niepert 2018;Dasgupta, Ray, and Talukdar 2018;Leblay and Chekol 2018). Furthermore, many use inadequate datasets such as YAGO2 (Hoffart et al. 2013), which are sparse in the time domain, or a time augmented version of FreeBase (Bollacker et al. 2008), where time is appended to some existing facts. ...
Article
Despite the importance and abundance of temporal knowledge graphs, most of the current research has been focused on reasoning on static graphs. In this paper, we study the challenging problem of inference over temporal knowledge graphs. In particular, the task of temporal link prediction. In general, this is a difficult task due to data non-stationarity, data heterogeneity, and its complex temporal dependencies. We propose Chronological Rotation embedding (ChronoR), a novel model for learning representations for entities, relations, and time. Learning dense representations is frequently used as an efficient and versatile method to perform reasoning on knowledge graphs. The proposed model learns a k-dimensional rotation transformation parametrized by relation and time, such that after each fact's head entity is transformed using the rotation, it falls near its corresponding tail entity. By using high dimensional rotation as its transformation operator, ChronoR captures rich interaction between the temporal and multi-relational characteristics of a Temporal Knowledge Graph. Experimentally, we show that ChronoR is able to outperform many of the state-of-the-art methods on the benchmark datasets for temporal knowledge graph link prediction.
... Know-Evolve [120] learns non-linearly evolving entity representations at different times based on a novel deep evolutionary knowledge network, and it utilises the temporal point process to describe the impact of different time points. TTransE [121] extends TransE to encode time information in the same representation space as entities and relations, with the scoring function defined as ...
Preprint
Full-text available
Knowledge Graphs (KGs) which can encode structural relations connecting two objects with one or multiple related attributes have become an increasingly popular research direction. Given the superiority of deep learning in representing complex data in continuous space, it is handy to represent KGs data, thus promoting KGs construction, representation, and application. This survey article provides a comprehensive overview of deep learning technologies and KGs by exploring research topics from diverse phases of the KGs lifecycle, such as construction, representation, and knowledge-aware application. We propose new taxonomies on these research topics for motivating cross-understanding between deep learning and KGs. Based on the above three phases, we classify the different tasks of KGs and task-related methods. Afterwards, we explain the principles of combing deep learning in various KGs steps like KGs embedding. We further discuss the contribution and advantages of deep learning applied to the different application scenarios. Finally, we summarize some critical challenges and open issues deep learning approaches face in KGs.
... Furthermore, for the final scoring of the newly predicted facts, it employs TransE on the projected embeddings. There are several other TKGEs which have been proposed as extensions of TransE such as TTransE [14] and TA-TransE [6]. HyTE and other extensions do not consider any hypercomplex algebraic aspects that could let the model cover the spatial information beside the temporal ones. ...
Chapter
Many knowledge graphs (KG) contain spatial and temporal information. Most KG embedding models follow triple-based representation and often neglect the simultaneous consideration of the spatial and temporal aspects. Encoding such higher dimensional knowledge necessitates the consideration of true algebraic and geometric aspects. Hypercomplex algebra provides the foundation of a well defined mathematical system among which the Dihedron algebra with its rich framework is suitable to handle multidimensional knowledge. In this paper, we propose an embedding model that uses Dihedron algebra for learning such spatial and temporal aspects. The evaluation results show that our model performs significantly better than other adapted models.
... Each fact takes the form of triplet (ℎ, , ), denoting that a head entity ℎ has a relation with a tail entity . There have been several large scale knowledge graphs such as WordNet [28], DBpedia [3], YAGO [6,34], FreeBase [7] and NELL [11], in which the structured information benefits a wide range of artificial intelligence applications including question answering [8,16,21], event prediction [25,42] and recommender systems [10,45]. ...
Preprint
Knowledge graph embedding (KGE) has been intensively investigated for link prediction by projecting entities and relations into continuous vector spaces. Current popular high-dimensional KGE methods obtain quite slight performance gains while require enormous computation and memory costs. In contrast to high-dimensional KGE models, training low-dimensional models is more efficient and worthwhile for better deployments to practical intelligent systems. However, the model expressiveness of semantic information in knowledge graphs (KGs) is highly limited in the low dimension parameter space. In this paper, we propose iterative self-semantic knowledge distillation strategy to improve the KGE model expressiveness in the low dimension space. KGE model combined with our proposed strategy plays the teacher and student roles alternatively during the whole training process. Specifically, at a certain iteration, the model is regarded as a teacher to provide semantic information for the student. At next iteration, the model is regard as a student to incorporate the semantic information transferred from the teacher. We also design a novel semantic extraction block to extract iteration-based semantic information for the training model self-distillation. Iteratively incorporating and accumulating iteration-based semantic information enables the low-dimensional model to be more expressive for better link prediction in KGs. There is only one model during the whole training, which alleviates the increase of computational expensiveness and memory requirements. Furthermore, the proposed strategy is model-agnostic and can be seamlessly combined with other KGE models. Consistent and significant performance gains in experimental evaluations on four standard datasets demonstrate the effectiveness of the proposed self-distillation strategy.
Article
Knowledge Graph (KG) provides high-quality structured knowledge for various downstream knowledge-aware tasks (such as recommendation and intelligent question-answering) with its unique advantages of representing and managing massive knowledge. The quality and completeness of KGs largely determine the effectiveness of the downstream tasks. But in view of the incomplete characteristics of KGs, there is still a large amount of valuable knowledge is missing from the KGs. Therefore, it is necessary to improve the existing KGs to supplement the missed knowledge. Knowledge Graph Completion (KGC) is one of the popular technologies for knowledge supplement. Accordingly, there has a growing concern over the KGC technologies. Recently, there have been lots of studies focusing on the KGC field. To investigate and serve as a helpful resource for researchers to grasp the main ideas and results of KGC studies, and further highlight ongoing research in KGC, in this paper, we provide a all-round up-to-date overview of the current state-of-the-art in KGC. According to the information sources used in KGC methods, we divide the existing KGC methods into two main categories: the KGC methods relying on structural information and the KGC methods using other additional information. Further, each category is subdivided into different granularity for summarizing and comparing them. Besides, the other KGC methods for KGs of special fields (including temporal KGC, commonsense KGC, and hyper-relational KGC) are also introduced. In particular, we discuss comparisons and analyses for each category in our overview. Finally, some discussions and directions for future research are provided.
Article
This paper proposes a class of temporal association rules, denoted by TACOs, for event prediction. As opposed to previous graph rules, TACOs monitor updates to graphs, and can be used to capture temporal interests in recommendation and catch frauds in response to behavior changes, among other things. TACOs are defined on temporal graphs in terms of change patterns and (temporal) conditions, and may carry machine learning (ML) predicates for temporal event prediction. We settle the complexity of reasoning about TACOs, including their satisfiability, implication and prediction problems. We develop a system, referred to as TASTE. TASTE discovers TACOs by iteratively training a rule creator based on generative ML models in a creator-critic framework. Moreover, it predicts events by applying the discovered TACOs. Using real-life and synthetic datasets, we experimentally verify that TASTE is on average 31.4 times faster than conventional data mining methods in TACO discovery, and it improves the accuracy of state-of-the-art event prediction models by 23.4%.
Article
To solve the incompleteness problem in temporal knowledge graphs (TKGs) and to discover the new knowledge, TKG completion remains an essential task always solved by graph embedding technology. Existing TKG completion methods encode time at only single granularity, which is insufficient in exploiting the rich information of distinct time granularities. Furthermore, most of them lack a comprehensive consideration of the characteristic of both time points and time periods, resulting in the inability to handle the two types of facts with different time forms, namely the discrete facts and continuous facts, simultaneously. In this paper, we propose a novel TKG embedding model which introduces the block term tensor decomposition and utilizes the core tensor and factor matrices to capture information presented by facts under distinct time granularities. By focusing on moments included in the time period and treating the discrete fact as a special case of the continuous fact, the model manages the processing of different types of facts in a unified manner. Besides, we explicitly design the static properties of entities and relations as well as their interactions to conform to reality. Experiments on 3 real datasets of different types verify the effectiveness of our proposed method compared with most state-of-the-art methods.
Article
Knowledge graph question answering (KGQA) has recently received a lot of attention and many innovative methods have been proposed in this area, but few have been developed for temporal KGQA. Most of the existing temporal KGQA methods focus on semantic or temporal level matching and lack the ability to reason about time constraints. In this paper we propose a subgraph-based model for answering complex questions over temporal knowledge graphs (TKG), inspired by human cognition. Our method, called SubGraph Temporal Reasoning (SubGTR), consists of three main modules: implicit knowledge extraction, relevant facts search, and subgraph logic reasoning. First, the question is reformulated using background knowledge stored in the temporal knowledge graph to acquire explicit time constraints. Then, the TKG is being searched to identify relevant entities and obtain an initial scoring of them. Finally the time constraints are quantified and applied using temporal logic to reach to the final answer. To evaluate our model we experiment against temporal QA benchmarks. We observe that existing benchmarks contain many pseudo-temporal questions, and we propose Complex-CronQuestions, which a filtered version of CronQuestions and which can better demonstrate the model’s inference ability for complex temporal questions. Experimental results show that SubGTR achieves state-of-the-art performance on both CronQuestions and Complex-CronQuestions. Moreover, our model shows better performance in handling the entity cold-start problem compared to existing temporal KGQA methods.
Article
Reasoning on knowledge graph (KG) has been studied for explainable recommendation due to its ability of providing explicit explanations. However, current KG-based explainable recommendation methods unfortunately ignore the temporal information (such as purchase time, recommend time, etc. ), which may result in unsuitable explanations. In this work, we propose a novel Time-aware Path reasoning for Recommendation (TPRec for short) method, which leverages the potential of temporal information to offer better recommendation with plausible explanations. First, we present an efficient time-aware interaction relation extraction component to construct collaborative knowledge graph with time-aware interactions (TCKG for short), and then introduce a novel time-aware path reasoning method for recommendation. We conduct extensive experiments on three real-world datasets. The results demonstrate that the proposed TPRec could successfully employ TCKG to achieve substantial gains and improve the quality of explainable recommendation.
Preprint
Full-text available
Temporal knowledge prediction is a crucial task for the event early warning that has gained increasing attention in recent years, which aims to predict the future facts by using relevant historical facts on the temporal knowledge graphs. There are two main difficulties in this prediction task. First, from the historical facts point of view, how to model the evolutionary patterns of the facts to predict the query accurately. Second, from the query perspective, how to handle the two cases where the query contains seen and unseen entities in a unified framework. Driven by the two problems, we propose a novel adaptive pseudo-siamese policy network for temporal knowledge prediction based on reinforcement learning. Specifically, we design the policy network in our model as a pseudo-siamese policy network that consists of two sub-policy networks. In sub-policy network I, the agent searches for the answer for the query along the entity-relation paths to capture the static evolutionary patterns. And in sub-policy network II, the agent searches for the answer for the query along the relation-time paths to deal with unseen entities. Moreover, we develop a temporal relation encoder to capture the temporal evolutionary patterns. Finally, we design a gating mechanism to adaptively integrate the results of the two sub-policy networks to help the agent focus on the destination answer. To assess our model performance, we conduct link prediction on four benchmark datasets, the experimental results demonstrate that our method obtains considerable performance compared with existing methods.
Chapter
Representation learning for the Temporal Knowledge Graphs (TKGs) is an emerging topic in the knowledge reasoning community. Existing methods consider the internal and external influence at either element level or fact level. However, the multi-granularity information is essential for TKG modeling and the connection in between is also under-explored. In this paper, we propose the method that Aligning-internal Regularity and external Influence of Multi-granularity for Temporal knowledge graph Embedding (ARIM-TE). In particular, to prepare considerate source information for alignment, ARIM-TE first models element-level information via the addition between internal regularity and the external influence. Based on the element-level information, the merge gate is introduced to model the fact-level information by combining their internal regularity including the local and global influence with external random perturbation. Finally, according to the above obtained multi-granular information of rich features, ARIM-TE conducts alignment for them in both structure and semantics. Experimental results show that ARIM-TE outperforms current state-of-the-art KGE models on several TKG link prediction benchmarks.
Article
Knowledge graph (KG) has increasingly been seen as a significant resource in financial applications (e.g., risk control, auditing and anti-fraud). However, there are few prior studies that focus on multi-relational circles, extracting additional information under the completed KG and selecting similarity measures for knowledge representation. In this paper, we introduce multi-relational circles and propose a novel embedding model, which considers entity weights calculated by PageRank algorithm to improve TransE method. In order to extract additional information, we use entity weights to convert embeddings into an on-map mining problem, and propose a model called CNNe based on entity weights and a convolutional neural network with three hidden layers, which converts vectors of entities, entity weights and relationships into matrices to perform link prediction in the same way as image processing. With the help of ten different similarity measures, it is demonstrated that the choice of distance measure greatly effect the results of the translation embedding models. Moreover, we propose two embedding methods, sMFE and tMFE, to enhance the results using matrix factorization. The complete incidence matrix is first applied to knowledge embedding, which contains the most comprehensive topological properties of the graph. Experimental results on standard benchmark datasets demonstrate that the proposed models are effective. In particular, CNNe achieves a mean rank of 166 less than the baseline method and an improvement of 2.1% on the proportion of correct entities ranked in the top ten on YAGO3-10 dataset.
Conference Paper
Full-text available
The major components of the brain’s declarative or explicit memory are semantic memory and episodic memory. Whereas semantic memory stores general factual knowledge, episodic memory stores events together with their temporal and spatial contexts. We present mathematical models for declarative memories where we consider semantic memory to be represented by triples and episodes to be represented as quadruples i.e., triples in time. E.g., (Jack, receivedDiagnosis, Diabetes, Jan1) states that Jack was diagnosed with diabetes on January 1. Both from a cognitive and a technical perspective, an interesting research question is how declarative data can efficiently be stored and semantically be decoded. We propose that a suitable data representation for episodic event data is a 4-way tensor with dimensions subject, predicate, object, and time. We demonstrate that the 4-way tensor can be decomposed, e.g., using a 4-way Tucker model, which permits semantic decoding of an event, as well as efficient storage. We also propose that semantic memory can be derived from the episodic model by a marginalization of the time dimension, which can be performed efficiently. We argue that the storage of episodic memory typically requires models with a high rank, whereas semantic memory can be modelled with a comparably lower rank. We analyse experimentally the relationship between episodic and semantic memory models and discuss potential relationships to the corresponding brain’s cognitive memories.
Article
Full-text available
Knowledge bases of real-world facts about entities and their relationships are useful resources for a variety of natural language processing tasks. However, because knowledge bases are typically incomplete, it is useful to be able to perform knowledge base completion, i.e., predict whether a relationship not in the knowledge base is likely to be true. This article presents an overview of embedding models of entities and relationships for knowledge base completion, with up-to-date experimental results on two standard evaluation tasks of link prediction (i.e. entity prediction) and triple classification.
Article
Full-text available
Learning embeddings of entities and relations is an efficient and versatile method to perform machine learning on relational data such as knowledge graphs. In this work, we propose holographic embeddings (HolE) to learn compositional vector space representations of entire knowledge graphs. The proposed method is related to holographic models of associative memory in that it employs circular correlation to create compositional representations. By using correlation as the compositional operator HolE can capture rich interactions but simultaneously remains efficient to compute, easy to train, and scalable to very large datasets. In extensive experiments we show that holographic embeddings are able to outperform state-of-the-art methods for link prediction in knowledge graphs and relational learning benchmark datasets.
Conference Paper
Full-text available
Information on the temporal interval of validity for facts de- scribed by RDF triples plays an important role in a large number of ap- plications. Yet, most of the knowledge bases available on the Web of Data do not provide such information in an explicit manner. In this paper, we present a generic approach which addresses this drawback by inserting temporal information into knowledge bases. Our approach combines two types of information to associate RDF triples with time intervals. First, it relies on temporal information gathered from the document Web by an extension of the fact validation framework DeFacto. Second, it harnesses the time information contained in knowledge bases. This knowledge is combined within a three-step approach which comprises the steps match- ing, selection and merging. We evaluate our approach against a corpus of facts gathered from Yago2 by using DBpedia and Freebase as input and different parameter settings for the underlying algorithms. Our results suggest that we can detect temporal information for facts from DBpedia with an F-measure of up to 70%.
Conference Paper
Full-text available
We consider here the problem of building a never-ending language learner; that is, an intelligent computer agent that runs forever and that each day must (1) extract, or read, information from the web to populate a growing structured knowledge base, and (2) learn to perform this task better than on the previous day. In particular, we propose an approach and a set of design principles for such an agent, describe a partial implementation of such a system that has already learned to extract a knowledge base containing over 242,000 beliefs with an estimated precision of 74% after running for 67 days, and discuss lessons learned from this preliminary attempt to build a never-ending learning agent. Copyright © 2010, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.
Conference Paper
Full-text available
DBpedia is a community effort to extract structured information from Wikipedia and to make this information available on the Web. DBpedia allows you to ask sophisticated queries against datasets derived from Wikipedia and to link other datasets on the Web to Wikipedia data. We describe the extraction of the DBpedia datasets, and how the resulting information is published on the Web for human- and machine-consumption. We describe some emerging applications from the DBpedia community and show how website authors can facilitate DBpedia content within their sites. Finally, we present the current status of interlinking DBpedia with other open datasets on the Web and outline how DBpedia could serve as a nucleus for an emerging Web of open data.
Conference Paper
Full-text available
Relational learning is becoming increasingly important in many areas of application. Here, we present a novel approach to relational learning based on the factorization of a three-way tensor. We show that unlike other tensor approaches, our method is able to perform collective learning via the latent components of the model and provide an efficient algorithm to compute the factorization. We substantiate our theoretical considerations regarding the collective learning capabilities of our model by the means of experiments on both a new dataset and a dataset commonly used in entity resolution. Furthermore, we show on common benchmark datasets that our approach achieves better or on-par results, if compared to current state-of-the-art relational learning solutions, while it is significantly faster to compute.
Conference Paper
Full-text available
We present YAGO2, an extension of the YAGO knowledge base with focus on temporal and spatial knowledge. It is automatically built from Wikipedia, GeoNames, and WordNet, and contains nearly 10 million entities and events, as well as 80 million facts representing general world knowledge. An enhanced data representation introduces time and location as first-class citizens. The wealth of spatio-temporal information in YAGO can be explored either graphically or through a special time- and space-aware query language.
Conference Paper
A common problem in knowledge representation and related fields is reasoning over a large joint knowledge graph, represented as triples of a relation between two entities. The goal of this paper is to develop a more powerful neural network model suitable for inference over these relationships. Previous models suffer from weak interaction between entities or simple linear projection of the vector space. We address these problems by introducing a neural tensor network (NTN) model which allow the entities and relations to interact multiplicatively. Additionally, we observe that such knowledge base models can be further improved by representing each entity as the average of vectors for the words in the entity name, giving an additional dimension of similarity by which entities can share statistical strength. We assess the model by considering the problem of predicting additional true relations between entities given a partial knowledge base. Our model outperforms previous models and can classify unseen relationships in WordNet and FreeBase with an accuracy of 86.2% and 90.0%, respectively.
Article
Knowledge bases are an important resource for question answering and other tasks but often suffer from incompleteness and lack of ability to reason over their discrete entities and relationships. In this paper we introduce an expressive neural tensor network suitable for reasoning over relationships between two entities. Previous work represented entities as either discrete atomic units or with a single entity vector representation. We show that performance can be improved when entities are represented as an average of their constituting word vectors. This allows sharing of statistical strength between, for instance, facts involving the "Sumatran tiger" and "Bengal tiger." Lastly, we demonstrate that all models improve when these word vectors are initialized with vectors learned from unsupervised large corpora. We assess the model by considering the problem of predicting additional true relations between entities given a subset of the knowledge base. Our model outperforms previous models and can classify unseen relationships in WordNet and FreeBase with an accuracy of 86.2% and 90.0%, respectively.
Article
Wikidata allows every user to extend and edit the stored information, even without creating an account. A form based interface makes editing easy. Wikidata's goal is to allow data to be used both in Wikipedia and in external applications. Data is exported through Web services in several formats, including JavaScript Object Notation, or JSON, and Resource Description Framework, or RDF. Data is published under legal terms that allow the widest possible reuse. The value of Wikipedia's data has long been obvious, with many efforts to use it. The Wikidata approach is to crowdsource data acquisition, allowing a global community to edit the data. This extends the traditional wiki approach of allowing users to edit a website. In March 2013, Wikimedia introduced Lua as a scripting language for automatically creating and enriching parts of articles. Lua scripts can access Wikidata, allowing Wikipedia editors to retrieve, process, and display data. Many other features were introduced in 2013, and development is planned to continue for the foreseeable future.
Article
We consider the problem of embedding entities and relationships of multi relational data in low-dimensional vector spaces. Our objective is to propose a canonical model which is easy to train, contains a reduced number of parameters and can scale up to very large databases. Hence, we propose TransE, a method which models relationships by interpreting them as translations operating on the low-dimensional embeddings of the entities. Despite its simplicity, this assumption proves to be powerful since extensive experiments show that TransE significantly outperforms state-of-the-art methods in link prediction on two knowledge bases. Besides, it can be successfully trained on a large scale data set with 1M entities, 25k relationships and more than 17M training samples.
Article
Recent years have witnessed a proliferation of large-scale knowledge bases, including Wikipedia, Freebase, YAGO, Microsoft's Satori, and Google's Knowledge Graph. To increase the scale even further, we need to explore automatic methods for constructing knowledge bases. Previous approaches have primarily focused on text-based extraction, which can be very noisy. Here we introduce Knowledge Vault, a Web-scale probabilistic knowledge base that combines extractions from Web content (obtained via analysis of text, tabular data, page structure, and human annotations) with prior knowledge derived from existing knowledge repositories. We employ supervised machine learning methods for fusing these distinct information sources. The Knowledge Vault is substantially bigger than any previously published structured knowledge repository, and features a probabilistic inference system that computes calibrated probabilities of fact correctness. We report the results of multiple studies that explore the relative utility of the different information sources and extraction methods.
Article
Factorization approaches provide high accuracy in several important prediction problems, for example, recommender systems. However, applying factorization approaches to a new prediction problem is a nontrivial task and requires a lot of expert knowledge. Typically, a new model is developed, a learning algorithm is derived, and the approach has to be implemented. Factorization machines (FM) are a generic approach since they can mimic most factorization models just by feature engineering. This way, factorization machines combine the generality of feature engineering with the superiority of factorization models in estimating interactions between categorical variables of large domain. libFM is a software implementation for factorization machines that features stochastic gradient descent (SGD) and alternating least-squares (ALS) optimization, as well as Bayesian inference using Markov Chain Monto Carlo (MCMC). This article summarizes the recent research on factorization machines both in terms of modeling and learning, provides extensions for the ALS and MCMC algorithms, and describes the software tool libFM.
Conference Paper
Recommender systems provide users with personalized suggestions for products or services. These systems often rely on Collaborating Filtering (CF), where past transactions are analyzed in order to establish connections between users and products. The two more successful approaches to CF are latent factor models, which directly profile both users and products, and neighborhood models, which analyze similarities between products or users. In this work we introduce some innovations to both approaches. The factor and neighborhood models can now be smoothly merged, thereby building a more accurate combined model. Further accuracy improvements are achieved by extending the models to exploit both explicit and implicit feedback by the users. The methods are tested on the Netflix data. Results are better than those previously published on that dataset. In addition, we suggest a new evaluation metric, which highlights the differences among methods, based on their performance at a top-K recommendation task.
Conference Paper
Recent research has made significant advances in automatically constructing knowledge bases by extracting relational facts (e.g., Bill Clinton-presidentOf-US) from large text corpora. Temporally scoping such relational facts in the knowledge base (i.e., determining that Bill Clinton-presidentOf-US is true only during the period 1993 - 2001) is an important, but relatively unexplored problem. In this paper, we propose a joint inference framework for this task, which leverages fact-specific temporal constraints, and weak supervision in the form of a few labeled examples. Our proposed framework, CoTS (Coupled Temporal Scoping), exploits temporal containment, alignment, succession, and mutual exclusion constraints among facts from within and across relations. Our contribution is multi-fold. Firstly, while most previous research has focused on micro-reading approaches for temporal scoping, we pose it in a macro-reading fashion, as a change detection in a time series of facts' features computed from a large number of documents. Secondly, to the best of our knowledge, there is no other work that has used joint inference for temporal scoping. We show that joint inference is effective compared to doing temporal scoping of individual facts independently. We conduct our experiments on large scale open-domain publicly available time-stamped datasets, such as English Gigaword Corpus and Google Books Ngrams, demonstrating CoTS's effectiveness.
Conference Paper
Tagging plays an important role in many recent websites. Recommender systems can help to suggest a user the tags he might want to use for tagging a specific item. Factorization models based on the Tucker Decomposition (TD) model have been shown to provide high quality tag recommendations outperforming other approaches like PageRank, FolkRank, collaborative filtering, etc. The problem with TD models is the cubic core tensor resulting in a cubic runtime in the factorization dimension for prediction and learning. In this paper, we present the factorization model PITF (Pairwise Interaction Tensor Factorization) which is a special case of the TD model with linear runtime both for learning and prediction. PITF explicitly models the pairwise interactions between users, items and tags. The model is learned with an adaption of the Bayesian personalized ranking (BPR) criterion which originally has been introduced for item recommendation. Empirically, we show on real world datasets that this model outperforms TD largely in runtime and even can achieve better prediction quality. Besides our lab experiments, PITF has also won the ECML/PKDD Discovery Challenge 2009 for graph-based tag recommendation.
Conference Paper
ASALSAN is a new algorithm for computing three-way DEDICOM, which is a linear algebra model for analyzing intrinsically asymmetric relationships, such as trade among nations or the exchange of emails among individuals, that incorporates a third mode of the data, such as time. ASALSAN is unique because it enables computing the three-way DEDICOM model on large, sparse data. A nonnegative version of ASALSAN is described as well. When we apply these techniques to adjacency arrays arising from directed graphs with edges labeled by time, we obtain a smaller graph on latent semantic dimensions and gain additional information about their changing relationships over time. We demonstrate these techniques on international trade data and the Enron email corpus to uncover latent components and their transient behavior. The mixture of roles assigned to individuals by ASALSAN showed strong correspondence with known job classifications and revealed the patterns of communication between these roles. Changes in the communication pattern over time, e.g., between top executives and the legal department, were also apparent in the solutions.
Translating embeddings for modeling multi-relational data Advances in neural information processing systems
  • Antoine Bordes
  • Nicolas Usunier
  • Alberto Garcia-Duran
  • Jason Weston
  • Oksana Yakhnenko
Towards Time-Aware Knowledge Graph Completion
  • Tingsong Jiang
  • Tianyu Liu
  • Tao Ge
  • Lei Sha
  • Baobao Chang
  • Sujian Li
  • Zhifang Sui
Tingsong Jiang, Tianyu Liu, Tao Ge, Lei Sha, Baobao Chang, Sujian Li, and Zhifang Sui. 2016. Towards Time-Aware Knowledge Graph Completion.. In COLING. 1715-1724.
Hybrid acquisition of temporal scopes for rdf data European Semantic Web Conference
  • Anisa Rula
  • Matteo Palmonari
  • Axel-Cyrille Ngonga Ngomo
  • Daniel Gerber
  • Jens Lehmann
  • Lorenz Bühmann
DBpedia: a nucleus for a web of open data, Proceedings of the 6th international The semantic web and 2nd Asian conference on Asian semantic web conference
  • Sören Auer
  • Christian Bizer
  • Georgi Kobilarov
  • Jens Lehmann
  • Richard Cyganiak
  • Zachary Ives
Embedding Learning for Declarative Memories
  • Yunpu Volker Tresp
  • Stephan Ma
  • Yinchong Baier
  • Yang
Dbpedia: A nucleus for a web of open data. The semantic web
  • Sören Auer
  • Christian Bizer
  • Georgi Kobilarov
  • Jens Lehmann
  • Richard Cyganiak
  • Zachary Ives
  • Rakshit Trivedi
  • Mehrdad Farajtabar
  • Yichen Wang
  • Hanjun Dai
  • Hongyuan Zha
  • Le Song
Rakshit Trivedi, Mehrdad Farajtabar, Yichen Wang, Hanjun Dai, Hongyuan Zha, and Le Song. 2017. Know-Evolve: Deep Reasoning in Temporal Knowledge Graphs. arXiv preprint arXiv:1705.05742 (2017).