Translating embeddings for modelling multi-relational data

Contribution

Proposed TransE, which models relationships as translation operations on low-dimensional entities embeddings. There has always been a trade-off between expressivity of models and complexity (accuracy vs scalability). The main motivation behind TransE is that hierarchical relationships are common between entities and therefore translations are good transformations to represent them! In addition, TransE relies on lower number of parameters (scalability).

Future Work

  1. Investigate if all relationship types (1-1,1-many, many-1) can be effectively model using this approach

  2. Combining KBs with text, shown promising results when applied to relation extraction from text!

Learning structured embeddings of knowledge bases

Contribution

Using neural network architecture to represent symbolic representations of KBs into low dimensional embedding vector space. The model would learn one embedding for each entity and one operator (matrix) for each relation!

Future Work

  1. Learn a combined knowledge set in a single embedding space from multi-tasking in different KBs

  2. Insert this new knowledge into different AI tasks to see if there are any improvements

Complex embeddings for simple link prediction

Contribution

Use complex valued embeddings for link prediction. Complex embeddings can handle a variety of binary relations including symmetric and antisymmetric relations. The complex embeddings is also relatively simpler and highly scalable to large knowledge graphs!

Future Work

  1. Merge complex embeddings with tensor factorisation approach

  2. Develop a better negative sampling procedure to generate higher quality negative samples, which could speed up training time

A semantic matching energy function for learning with multi-relational data

Contribution

Proposed the semantic matching energy function to encode multi-relational graph into low dimensional vectors that capture the semantic meaning of the relations as well as using it to compute similarities between entities and relations.

Ryan

Ryan

Data Scientist

Leave a Reply