RotatE: Knowledge graph embedding by relational rotation in complex space

Proposed RotatE for knowledge graph embeddings that can model and infer different relation patterns such as symmetry, inversion, and composition. RotatE is consider to be the first model to model all relation patterns! RotatE model defines each relation as rotation transformation from source entity to target entity in the complex vector space. The paper also proposed a novel negative sampling technique to effectively train the RotatE model! The model was applied to link prediction and outperformed SOTA models. The model is linear in time and space complexity, therefore, scalable to large knowledge graphs.

Future Work

  1. Evaluate RotatE model to more datasets and leverage a probabilistic framework to model uncertainties of entities and relations

A three-way model for collective learning on multi-relational data

Proposed RESCAL, a three-way tensor factorisation method, for relational learning. The reasons for using tensor factorisation is that a) simplicity in modelling multiple relations of any order using high-order tensor b) factorisation methods tend to do well in high-dimensional and sparse domain, which relational falls under! c) Capturing the correlations between multiple interconnected nodes using attributes, relations or classes of related entities in a learning task.

Future Work

  1. Work on distributed RESCAL and stochastic gradient descent approach to optimisation problem to improve the scalability of RESCAL

  2. Use constraints during factorisation to improve the predictive performance and run-time behaviour of RESCAL

SimplE embedding for link prediction in knowledge graphs

Proposed SimplE, an enhancement of Canonical Polyadic (CP), which it’s one of the first tensor factorisation approaches. The CP method learns one embedding vector for each relation and TWO embedding vectors for each entity. one when the entity is a head entity and the other when the entity is a tail entity. However, the head entity embedding is learned independently of the tail entity embedding, which has resulted in poor performance in KG completion. SimplE aims to solve this independence issue and the paper show that SimplE has the following attribute:

  1. Can be seen as a bilinear model

  2. Fully expressive

  3. Can encode background knowledge into its embedding through weight sharing

Future Work

  1. Build ensembles of SimplE models

  2. Add SimplE to relation-level ensembles

  3. Explicitly model the analogical structures of relations

  4. Use 1-N scoring method to generate negative samples

  5. Combine SimplE with logic-based methods

  6. Incorporate other auxiliary information into the SimplE embeddings

Ryan

Ryan

Data Scientist

Leave a Reply