What are the two chatbot architectures?
Rule-based systems – ELIZA and PARRY
Corpus-based systems – uses large datasets of conversations between humans
ELIZA was designed to simulate Rogerian psychologist, a branch of clinical psychology that involves drawing a patient out by reflecting the patient’s conversation / statements back at them. Below is the most famous ELIZA conversation.
ELIZA is governed by pattern/ transform rules, transforming user inputs into appropriate relevant responses. Therefore, each ELIZA rule is connected to a keyword that might occur in a user’s utterance. The keywords are ranked, with specific words having higher ranks and general words having lower ranks. Below is the algorithm that ELIZA follows.
An example would be the following sentence “I know everybody thinks I am insane”. The word “I” has the transform rule of (I *) –> (You say you *), transforming the original sentence to “You say you know everybody thinks you are insane”. The problem here is that “I” is a very general word and so it could lead to general responses whereas the word “everybody” is more universals and we can assume that whatever it is, it’s referring th a specific event or person. Therefore, ELIZA have a tendency to select specific keywords rather than generic ones.
ELIZA consists of a memory trick that involves storing previous conversations and randomly select a transform on the MEMORY list. This happens when the highest ranked keyword is “my”.
How many words are estimated to be required to train a modern corpus-based chatbot?
Hundreds of millions – billions of words!
What is XiaoIce?
It’s a system that collects and stores all the human-machine conversations between XiaoIce and its users, with the purpose of using them as additional training data. XiaoIce has more than 30 billion conversation pairs.
What are the two main architectures for corpus-based chatbots?
Machine learned sequence transduction