Long-tail relation extraction via knowledge graph embeddings and graph convolution networks

Proposed a distance supervised relation extraction approach for long-tailed, imbalanced data by taking advantage of knowledge from data-rich classes at the head of distribution to boost performance of data-poor classes. This consists of two steps:

  1. Leverage implicit relational knowledge among class labels from knowledge graph embeddings and learn explicit relational knowledge using GCNs

  2. Integrate relational knowledge into relation extraction model using coarse-to-fine knowledge-aware attention mechanism

The two steps above solve the following problems:

  1. Learning relevant relational knowledge. Relevant relational knowledge would boost transfer knowledge and help with prediction of long-tail relations but irrelevant relational knowledge would result in negative knowledge transfer

  2. Integration of relational knowledge to existing RE models is challenging

Future Work

  1. Combine our method with denoising methods to further improve performance

  2. Combine rule mining and reasoning technologies to learn better class embeddings

  3. Apply our method to zero-shot RE and further adapt to other NLP tasks

Attention guided graph convolutional networks for relation extraction

Proposed AGGCNs, Attention Guided Graph Convolutional Networks, that directly takes full dependency trees as inputs. Most existing relation extraction models can be categorised into two classes: sequence-based and dependency-based. Sequence-based operate only on word sequences whereas dependency-based incorporate dependency trees into models. Our model learn a “soft pruning” strategy that transforms original dependency tree into a fully connected edge-weighted (strength of relatedness between nodes) graph. These weights are learned using the self-attention mechanism. With the help of dense connections, we can train a deep AGGCN model that can capture rich local and non-local dependency information. Compared to previous GCNs, our model can operate in parallel when parsing dependency trees and therefore doesn’t have additional computational cost.

Future Work

  1. Examine the use of proposed framework to improve graph representation learning for graph related tasks

Ryan

Ryan

Data Scientist

Leave a Reply