Knowledge vault: A web-scale approach to probabilistic knowledge fusion

Introduced Knowledge Vault, a web-scale approach to automate the construction of knowledge bases. This is aiming to solve the costly effort in manually building knowledge bases. The Knowledge Vault is substantially bigger than other published knowledge base and has an inference system that computes the probability that a fact is correct. The challenge with automation lies in producing noisy and unreliable facts! To tackle this, we leverage already-cataloged knowledge to build prior models of fact correctness. The contributions are as follows:

  1. Knowledge Vault is built using a new method of combining noisy web extractions with prior knowledge derived from existing knowledge bases

  2. KV is much bigger than other comparable KBs

  3. Performed detailed comparisons of different extraction methods in terms of quality and coverage

Future Work

  1. Continue to scale KV, to store more facts about the world and use this to enhance downstream applications

CoKE: Contextualized knowledge graph embedding

Proposed CoKE, contextualised knowledge graph embeddings, that learns dynamic entity and relation embeddings based on the context in which they appear in. They used two inputs for their model: edges and paths, both of which can be formulated as sequences of entities and relations. CoKE takes those inputs and use Transformer encoder to obtain the contextualised embeddings.

Future Work

  1. Generalise CoKE to different types of graph contexts (test out different inputs)

  2. Apply CoKE to more downstream tasks, both within KG and in broader domains

KG-BERT: BERT for knowledge graph completion

Proposed KG-BERT to model triples for knowledge graph completion. The model takes entity and relation descriptions of triples as input and computes a scoring function of the triple. The contributions are:

  1. Propose language modelling method for KGC. This is the first study to model triples’ plausibility with a pre-trained contextual language model

  2. Achieved SOTA results in triple classification, relation prediction, and link prediction tasks

Future Work

  1. Look into jointly model textual information with KG structures or utilise pre-trained models with more text data

  2. Apply KG-BERT to language understanding tasks and see if it leads to better performance

Ryan

Ryan

Data Scientist

Leave a Reply