- 微博 QQ QQ空间 贴吧
1 .The Rise of Conversational AI David Low Co-founder / CDS
2 .Bio ● Research Urban Mobility | Social Media ● Public Service GovTech(IDA) Data Science Division ● Teach Deep Learning Masterclass at National University of Singapore (NUS) ● Startup Conversational AI | Deep Natural Language Processing
3 .Overview ● Why Conversational AI? ● What is NLP? ● Current state of Conversational AI ● Why is NLP difficult? ● Open Source Framework ● Recent advancements ● What have we learnt? ● Demo of Question Answering System
4 .Rising trend of mobile (& messaging) usage
5 .As messaging apps have become indispensable parts of our lives, Enterprises are determined to be where their customers are.
6 .What is Natural Language Processing?
7 .What is Natural Language Processing? ● is a field at the intersection of Computer Science, Artificial Intelligence (AI) and Linguistics. ● Goal For computers to process or “understand” natural language in order to perform tasks that are useful ● Not to be confused with “Computational Linguistics” ● Deep NLP = Deep Learning based Natural Language Processing
8 .Example I - Sentiment Analysis
9 .Example II - Machine Translation Source: Google AI
10 .NLU vs NLP Source: Stanford NLP Group
11 .Current State of Conversational AI
12 .“While Siri, Alexa and the likes can follow simple spoken or typed commands and answer basic questions; they can’t hold a conversation and have no real understanding of the words they use. ” Will Knight
13 .In the past... Chatbot A Chatbot B
14 .Now Source: Salemove
15 .Why is NLP difficult? “Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo” - This is a grammatically correct sentence in American English.
16 .What is Homonym? (In linguistics) Words with identical pronunciation and spelling, whilst maintaining different meanings.
17 .Buffalo - a homonym 1 - Noun: The city of Buffalo, at western New York state, US. 2 - Noun: The animal, Buffalo. 3 - Verb: To confuse, deceive or intimidate.
18 .Three groups of buffalo “Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo”
19 .Why is NLP difficult? [cont] ● Sheer complexity of sentence structure ● Ambiguity ○ Eg: “I made her duck” ● Meaning is context sensitive ○ Depends on the people present e.g. “How far is it?” (miles, km?) ○ Depends on the time of day, e.g. “Let's go eat” ○ Depends on prior sentences: “The third one” ● Recognizing named entities (people/places/…) ● Slang, jargon, humour, sarcasm, spelling mistakes, grammar mistakes and abbreviations… Source: Stanford CS224
20 .Dialog research ● Purpose of language ○ Used to accomplish communication goals. ● Hence, solving dialog is a fundamental goal for NLP. ● Dialog can be seen as a single task (learning how to talk) OR as thousands of related tasks that require different skills, all using the same input and output format ● Example ○ Booking a restaurant, ○ Chatting about sports/news ○ Answering factual questions ○ …almost anything can be posed as Question Answering.
21 .Open Source Framework ● To provide an unified framework for the training and testing of dialog models ● Integration of Amazon Mechanical Turk & Facebook Messenger ● Multi-task training ● Datasets for over 20 tasks Source: FAIR
22 .ParlAI Tasks Source: FAIR
23 .SQuAD Tasks
24 .bAbI Tasks Source: FAIR
25 .bAbI Tasks Source: FAIR
26 .bAbI SOTA result GS Ramachandran et al 2017
27 .Visual Question Answering Source: FAIR
28 .Amazon Mechanical Turk Live Chat Source: FAIR
29 .Multi-Task Learning (MTL) ● Convention ML ○ Optimize for a particular metric (RMSE, AUC etc) ○ Train a single model or an ensemble of models ○ Fine-tune and tweak ● Multi-task Learning ○ Also known as “joint learning”, “learning to learn” etc ○ Optimize for more than one loss function ○ Why is it better? ■ It improves generalization by leveraging the domain-specific info contained in the training signals of related tasks.