展开查看详情
1 .The Rise of Conversational AI
David Low
Co-founder / CDS
2 .Bio
● Research
Urban Mobility | Social Media
● Public Service
GovTech(IDA) Data Science Division
● Teach
Deep Learning Masterclass at National University of Singapore (NUS)
● Startup
Conversational AI | Deep Natural Language Processing
3 .Overview
● Why Conversational AI?
● What is NLP?
● Current state of Conversational AI
● Why is NLP difficult?
● Open Source Framework
● Recent advancements
● What have we learnt?
● Demo of Question Answering System
4 .Rising trend of mobile (& messaging) usage
5 .As messaging apps have become indispensable parts of our lives, Enterprises
are determined to be where their customers are.
6 .What is Natural
Language Processing?
7 .What is Natural Language Processing?
● is a field at the intersection of Computer Science, Artificial
Intelligence (AI) and Linguistics.
● Goal
For computers to process or “understand” natural language in order to
perform tasks that are useful
● Not to be confused with “Computational Linguistics”
● Deep NLP = Deep Learning based Natural Language Processing
8 .Example I - Sentiment Analysis
9 .Example II - Machine Translation
Source: Google AI
10 .NLU vs NLP
Source: Stanford NLP Group
11 .Current State of
Conversational AI
12 .“While Siri, Alexa and the likes can follow simple spoken or typed commands
and answer basic questions; they can’t hold a conversation and have no real
understanding of the words they use. ”
Will Knight
13 .In the past...
Chatbot A Chatbot B
15 .Why is NLP difficult?
“Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo”
- This is a grammatically correct sentence in American English.
16 .What is Homonym?
(In linguistics) Words with identical
pronunciation and spelling, whilst maintaining
different meanings.
17 .Buffalo - a homonym
1 - Noun: The city of Buffalo, at western New York state, US.
2 - Noun: The animal, Buffalo.
3 - Verb: To confuse, deceive or intimidate.
18 .Three groups of buffalo
“Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo”
19 .Why is NLP difficult? [cont]
● Sheer complexity of sentence structure
● Ambiguity
○ Eg: “I made her duck”
● Meaning is context sensitive
○ Depends on the people present e.g. “How far is it?” (miles, km?)
○ Depends on the time of day, e.g. “Let's go eat”
○ Depends on prior sentences: “The third one”
● Recognizing named entities (people/places/…)
● Slang, jargon, humour, sarcasm, spelling mistakes, grammar mistakes
and abbreviations…
Source: Stanford CS224
20 .Dialog research
● Purpose of language
○ Used to accomplish communication goals.
● Hence, solving dialog is a fundamental goal for NLP.
● Dialog can be seen as a single task (learning how to talk) OR as
thousands of related tasks that require different skills, all using
the same input and output format
● Example
○ Booking a restaurant,
○ Chatting about sports/news
○ Answering factual questions
○ …almost anything can be posed as Question Answering.
21 .Open Source Framework
● To provide an unified framework for the training and testing of
dialog models
● Integration of Amazon Mechanical Turk & Facebook Messenger
● Multi-task training
● Datasets for over 20 tasks
Source: FAIR
22 .ParlAI Tasks
Source: FAIR
24 .bAbI Tasks
Source: FAIR
25 .bAbI Tasks
Source: FAIR
26 .bAbI SOTA result
GS Ramachandran et al 2017
27 .Visual Question Answering
Source: FAIR
28 .Amazon Mechanical Turk Live Chat
Source: FAIR
29 .Multi-Task Learning (MTL)
● Convention ML
○ Optimize for a particular metric (RMSE, AUC etc)
○ Train a single model or an ensemble of models
○ Fine-tune and tweak
● Multi-task Learning
○ Also known as “joint learning”, “learning to learn” etc
○ Optimize for more than one loss function
○ Why is it better?
■ It improves generalization by leveraging the domain-specific info
contained in the training signals of related tasks.
热门城市
