General Architecture of Neural MRC

A typical neural MRC takes in context and question and outputs the answer. It generally has 4 main modules: Embeddings, Feature Extraction, Context-Question Interaction, and Answer Prediction.

  • Embeddings
    • Convert text into fixed-length vectors in order for machine to understand natural language. Common embeddings include Word2Vec, GloVe, and Fasttext. Recent development on contextualised embeddings has shown to yield good improvement in encoding contextual information

  • Feature Extraction
    • In this module, we aim to extract more contextual information from the context and question embeddings. We can apply different deep neural networks such as RNNs and CNNs here

  • Context-Question Interaction
    • The relationship between the context and the question is important in generating the final answer. In this module, we want to find out which parts of the context is more relevant and significant to answering the question

    • Researchers have used attention mechanism to allow machine to focus on specific part of the context that’s relevant to the question

    • Sometimes, machine might need to “reread” the context multiple times to accurately form the answers. To mimic this, some research has introduce double encoders or decoders

  • Answer Prediction
    • This module is different depending on the type of MRC tasks. For cloze tests, this module will output a word or entity while for MCQ, the module will select the correct answer from the list of answers

    • For free answering task, some generation techniques are required as there are no constraint to the answer forms

Ryan

Ryan

Data Scientist

Leave a Reply