1.The Rise of Conversational AI David Low Co-founder / CDS
2.Bio ● Research Urban Mobility | Social Media ● Public Service GovTech(IDA) Data Science Division ● Teach Deep Learning Masterclass at National University of Singapore (NUS) ● Startup Conversational AI | Deep Natural Language Processing
3.Overview ● Why Conversational AI? ● What is NLP? ● Current state of Conversational AI ● Why is NLP difficult? ● Open Source Framework ● Recent advancements ● What have we learnt? ● Demo of Question Answering System
4.Rising trend of mobile (& messaging) usage
5.As messaging apps have become indispensable parts of our lives, Enterprises are determined to be where their customers are.
6.What is Natural Language Processing?
7.What is Natural Language Processing? ● is a field at the intersection of Computer Science, Artificial Intelligence (AI) and Linguistics. ● Goal For computers to process or “understand” natural language in order to perform tasks that are useful ● Not to be confused with “Computational Linguistics” ● Deep NLP = Deep Learning based Natural Language Processing
8.Example I - Sentiment Analysis
9.Example II - Machine Translation Source: Google AI
10.NLU vs NLP Source: Stanford NLP Group
11.Current State of Conversational AI
12.“While Siri, Alexa and the likes can follow simple spoken or typed commands and answer basic questions; they can’t hold a conversation and have no real understanding of the words they use. ” Will Knight
13.In the past... Chatbot A Chatbot B
14.Now Source: Salemove
15.Why is NLP difficult? “Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo” - This is a grammatically correct sentence in American English.
16.What is Homonym? (In linguistics) Words with identical pronunciation and spelling, whilst maintaining different meanings.
17.Buffalo - a homonym 1 - Noun: The city of Buffalo, at western New York state, US. 2 - Noun: The animal, Buffalo. 3 - Verb: To confuse, deceive or intimidate.
18.Three groups of buffalo “Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo”
19.Why is NLP difficult? [cont] ● Sheer complexity of sentence structure ● Ambiguity ○ Eg: “I made her duck” ● Meaning is context sensitive ○ Depends on the people present e.g. “How far is it?” (miles, km?) ○ Depends on the time of day, e.g. “Let's go eat” ○ Depends on prior sentences: “The third one” ● Recognizing named entities (people/places/…) ● Slang, jargon, humour, sarcasm, spelling mistakes, grammar mistakes and abbreviations… Source: Stanford CS224
20.Dialog research ● Purpose of language ○ Used to accomplish communication goals. ● Hence, solving dialog is a fundamental goal for NLP. ● Dialog can be seen as a single task (learning how to talk) OR as thousands of related tasks that require different skills, all using the same input and output format ● Example ○ Booking a restaurant, ○ Chatting about sports/news ○ Answering factual questions ○ …almost anything can be posed as Question Answering.
21.Open Source Framework ● To provide an unified framework for the training and testing of dialog models ● Integration of Amazon Mechanical Turk & Facebook Messenger ● Multi-task training ● Datasets for over 20 tasks Source: FAIR
22.ParlAI Tasks Source: FAIR
24.bAbI Tasks Source: FAIR
25.bAbI Tasks Source: FAIR
26.bAbI SOTA result GS Ramachandran et al 2017
27.Visual Question Answering Source: FAIR
28.Amazon Mechanical Turk Live Chat Source: FAIR
29.Multi-Task Learning (MTL) ● Convention ML ○ Optimize for a particular metric (RMSE, AUC etc) ○ Train a single model or an ensemble of models ○ Fine-tune and tweak ● Multi-task Learning ○ Also known as “joint learning”, “learning to learn” etc ○ Optimize for more than one loss function ○ Why is it better? ■ It improves generalization by leveraging the domain-specific info contained in the training signals of related tasks.