Robot Metaphors and Models
Braitenberg Vehicles and Quantum Automata Robots
Our Base Model and Designs
Probabilistic and Finite State Machines
Descriptions of Motions and Behaviors
Dialog and Robot’s Knowledge
1.Robot Metaphors and Models
2.Animatronic “Robot” or device brain effectors
3.Perceiving “Robot” sensors brain
4. Reactive Robot is the simplest behavioral robot Brain sensors is a effectors mapping This is the simplest robot that satisfies the definition of a robot
5. Reactive Robot in environment ENVIRONMENT is a feedback sensors brain effectors This is the simplest robot that satisfies the definition of a robot
6.Braitenberg Vehicles and Quantum Automata Robots
7.Another Example: Braitenberg Vehicles and Quantum BV
9. Emotional Robot has a simple form of memory or state Brain is a sensors Finite effectors State Machine This is the simplest robot that satisfies the definition of a robot
10. Behavior as an interpretation of a string • Newton, Einstein and Bohr. • Hello Professor • Hello Sir • Turn Left . Turn right. behavior
11. Behavior as an interpretation of a tree • Newton, Einstein and Bohr. • Hello Professor • Hello Sir • Turn Left . Turn right. behavior Grammar. Derivation. Alphabets.
12.Our Base Model and Designs
13. Fig. 1. Learning Behaviors as Mappings from environment’s features to interaction procedures probability Verbal response generation (text Speech from response and TTS). microphones Stored sounds Automatic software Head Image features construction from cameras movements from examples and facial (decision tree, bi bi-- emotions Sonars and other decomposition, sensors Ashenhurst,, DNF) Ashenhurst generation Neck Neck and shoulders and upper movement generation body movement Emotions and generation knowledge memory
14. Robot Head Construction, 1999 High school summer camps, hobby roboticists, undergraduates Furby head with new control Jonas We built and animated various kinds of humanoid heads with from 4 to 20 DOF, looking for comical and entertaining values.
15.Mister Butcher Latex skin from Hollywood 4 degree of freedom neck
16.Robot Head Construction, 2000 Skeleton Alien We use inexpensive servos from Hitec and Futaba, plastic, playwood and aluminum. The robots are either PC-interfaced, use simple micro-controllers such as Basic Stamp, or are radio controlled from a PC or by the user.
17.Technical Construction, 2001 Details Marvin the Crazy Robot Adam
18. Virginia Woolf 2001 heads equipped with microphones, USB cameras, sonars and CDS light sensors
19. 2002 Max BUG (Big Ugly Robot) Image processing and pattern recognition uses software developed at PSU, CMU and Intel (public domain software available on WWW). Software is in Visual C++, Visual Basic, Lisp and Prolog.
20. Visual Feedback and Learning based on Constructive Induction Uland Wong, 17 years old 2002
21. 2002, Japan Professor Perky Professor Perky with automated speech recognition (ASR) and text-to-speech (TTS) capabilities • We compared several commercial speech systems from Microsoft, Sensory and Fonix. •Based on experiences in highly noisy environments and with a variety of speakers, we selected Fonix for both ASR and TTS for Professor Perky and Maria robots. • We use microphone array 1 dollar latex skin from Andrea Electronics. from China
22. Maria, 2002/2003 20 DOF
23. Construction location details of Maria of head servos skull location of controlling rods location of remote Custom servos designed skin
24.Animation of eyes and eyelids
25. Cynthia, 2004, June
26. Currently the hands are not moveable. We have a separate hand design project.
27. Software/Hardware Architecture •Network- 10 processors, ultimately 100 processors. •Robotics Processors. ACS 16 •Speech cards on Intel grant •More cameras •Tracking in all robots. •Robotic languages – Alice and Cyc-like technologies.
28.Face detection localizes the person and is the first step for feature and face recognition. Acquiring information about the human: face detection and recognition, speech recognition and sensors.
29.Face features recognition and visualization.
30.Use of Multiple- Valued (five- valued) variables Smile, Mouth_Open and Eye_Brow_Raise for facial feature and face recognition.
31.HAHOE KAIST ROBOT THEATRE, KOREA, SUMMER 2004 Czy znacie dobra sztuke dla teatru robotow? Sonbi, the Confucian Scholar Paekchong, the bad butcher
33. Yangban the Aristocrat and Pune his concubine The Narrator
36.We base all our robots on inexpensive radio- controlled servo technology.
37. We are familiar with latex and polyester technologies for faces Martin Lukac and Jeff Allen wait for your help, whether you want to program, design behaviors, add muscles, improve vision, etc.
38.New Silicone Skins
39.A simplified diagram of software explaining the principle of using machine learning based on constructive induction to create new interaction modes of a human and a robot.
40.Probabilistic and Finite State Machines
41. Probabilistic State Machines to describe emotions “you are beautiful” P=1 / ”Thanks for a compliment” “you are blonde!” Happy state P=0.3 / ”I am not an idiot” “you are blonde!” / Do you suggest I am Unhappy state P=0.7 an idiot?” Ironic state
42. Facial Behaviors of Maria Maria asks: Do I look like younger than twenty three? Response: “no” “no” “yes” 0.7 0.3 Maria smiles Maria frowns
43. Probabilistic Grammars for performances Speak ”Professor Perky”, blinks eyes twice P=0.1 Speak ”Professor Perky” P=0.3 Where? Who? P=0.5 P=0.5 P=0.5 Speak “in some location”, smiles Speak “In the Speak ”Doctor Lee” broadly classroom”, shakes head P=0.1 What? Speak “Was P=0.1 P=0.1 P=0.1 Speak “Was singing and drinking wine” dancing” ….
44.Human-controlled modes of dialog/interaction Human teaches “Thanks, I “Hello Maria” have a lesson” “Lesson finished” Robot Robot asks “Question” performs “Stop performance” “Questioning “Command finished” finished” “Thanks, I “Thanks, I have a have a question” command” Human asks Human commands
45.Descriptions of Motions and Behaviors
46. Motion Descriptions • Two dimensional matrix. matrix 1. Rows are rotations of servos, 2. Column are robot poses. 3. Matrix is a robot gesture WAVEFORM 1 WAVEFORM 2 WAVEFORM 3 WAVEFORM N GESTURE 1 WE CAN USE ALGEBRA OF MATRICES, EIGENVALUES, GESTURE 2 EIGENVECTORS, MATRIX GESTURE 3 DECOMPOSITION ETC
47.Encoding poses by symbols A B AAB BABA You tall me?
48. Behavior Descriptions • Two dimensional matrix. matrix 1. Rows are conditions of executing this pose, 2. Column are robot poses. • Two dimensional matrix. matrix 3. Matrix is a robot gesture with conditions. 1. Rows are rotations of servos, 2. Column are robot poses. 3. Matrix is a robot gesture WAVEFORM 1 WAVEFORM 2 WAVEFORM 3 WAVEFORM N GESTURE 1 WE CAN USE ALGEBRA OF MATRICES, EIGENVALUES, GESTURE 2 EIGENVECTORS, MATRIX GESTURE 3 DECOMPOSITION ETC
49.• If A and B then do X1 and X3 A B Partial conditions C (Boolean) D A X1 B Partial conditions X2 Partial motions - (Boolean) (servos or servo X3 - sequences) X4 X1 previous Partial motions (servos or servo X3 sequences) - previous
50.• If A and C’ and D’ then do X1, X2 and X3 A B Partial conditions C (Boolean) D A Does X1 not care Partial conditions X2 Partial motions C’ (Boolean) (servos or servo X3 D’ sequences) X4 X1 X2 Partial motions (servos or servo X3 sequences) prev - ious
51. Behaviors as trees Braitenberg Vehicle Condition A Braitenberg Faces yes no Braitenberg TURN Bipeds RIGHT Condition B yes no Condition C yes no GO STRAIGHT TURN LEFT Go Back
52. Behaviors as strings TURN A RIGHT Condition A yes no GO A’B’ STRAIGHT TURN RIGHT Condition B A’ B C’ Go Back yes no Condition C A’ B C TURN LEFT yes no GO STRAIGHT condition Motion or TURN LEFT Go Back pose Condition can be generalized to a Boolean or Multivalued expression
53.Behaviors as state machines I am in a happy state Condition A yes no Condition B yes no Condition C yes STRAIGHTin an I am no GO TURN LEFT Go Back angry state
54.Dialog and Robot’s Knowledge
55.Robot-Receptionist Initiated Conversation Robot Human What can I do for you? Robot asks This represents operation mode
56.Robot-Receptionist Initiated Conversation Robot Human What can I do for you? I would like to order a table for two Robot asks
57.Robot-Receptionist Initiated Conversation Robot Human Smoking or non- smoking? Robot asks
58.Robot-Receptionist Initiated Conversation Robot Human Smoking or non- I do not understand smoking? Robot asks
59.Robot-Receptionist Initiated Conversation Robot Human Do you want a table in a smoking or non-smoking section of the restaurant? Non-smoking section is near the terrace. Robot asks
60.Robot-Receptionist Initiated Conversation Robot Human Do you want a table in a smoking or non-smoking A table near the section of the restaurant? terrace, please Non-smoking section is near the terrace. Robot asks
61.Human-Initiated Conversation Robot Human Hello Maria initialization Robot asks
62.Human-Initiated Conversation Robot Human Hello Maria What can I do for you? Robot asks
63. Human-Asking Robot Human Question Robot asks Question Human asks
64. Human-Asking Robot Human Yes, you ask a Question question. Human asks
65. Human-Asking Robot Human Yes, you ask a What book wrote Lee? question. Human asks
66. Human-Asking Robot Human I have no sure What book wrote Lee? information. Human asks
67. Human-Asking Robot Human I have no sure Try to guess. information. Human asks
68. Human-Asking Robot Human Lee wrote book Try to guess. “Flowers”. Human asks
69. Human-Asking Robot Human Lee wrote book This is not true. “Flowers”. Human asks
70. Human ends Human-Teaching questioning Robot Human Questioning finished Human asks Human teaches “Questioning finished” Robot asks Thanks, I have a lesson
71. Robot enters Human-Teaching asking mode Robot Human What can I do for you? Questioning finished Human asks Human teaches “Questioning finished” Robot asks Thanks, I have a lesson
72. Human starts Human-Teaching teaching Robot Human What can I do for you? Thanks, I have a lesson Human asks Human teaches “Questioning finished” Robot asks Thanks, I have a lesson
73. Human-Teaching Robot Human Yes Thanks, I have a lesson Human teaches
74. Human-Teaching Robot Human I give you question- Yes answer pattern Human teaches
75. Human-Teaching Robot Human Question pattern: Yes What book Smith wrote? Human teaches
76. Human-Teaching Robot Human Answer pattern: Yes Smith wrote book “Automata Theory” Human teaches
77. Human-Teaching Robot Human Checking question: Yes What book wrote Smith? Human teaches
78. Human-Teaching Robot Human Checking question: Smith wrote book What book wrote Smith? “Automata Theory” Human teaches
79. Human-Teaching Robot Human I give you question- Yes answer pattern Human teaches
80. Human-Teaching Robot Human Question pattern: Yes Where is room of Lee? Human teaches
81. Human-Teaching Robot Human Answer pattern: Yes Lee is in room 332 Human teaches
82.Human-Checking what robot learned Robot Human Lesson finished Human teaches “Lesson finished” Robot asks Question Human asks
83.Human-Checking what robot learned Robot Human What can I do for Lesson finished you? Human teaches “Lesson finished” Robot asks Question Human asks
84.Human-Checking what robot learned Robot Human What can I do for Question you? Human teaches “Lesson finished” Robot asks Question Human asks
85. Human-Asking Robot Human Yes, you ask a Question question. Human teaches “Lesson finished” Robot asks Question Human asks
86. Human-Asking Robot Human Yes, you ask a What book wrote Lee? question. Human asks
87. Human-Asking Robot Human I have no sure What book wrote Lee? information. Human asks
88. Human-Asking Robot Human I have no sure Try to guess. information. Human asks
89. Human-Asking Robot Human Lee wrote book Try to guess. “Automata Theory” Observe that robot found similarity between Smith and Lee and generalized Human asks (incorrectly)
90. Behavior, Dialog and Learning • The dialog/behavior has the following components: – (1) Eliza-like natural language dialogs based on pattern matching and limited parsing. • Commercial products like Memoni, Dog.Com, Heart, Alice, and Doctor all use this technology, very successfully – for instance Alice program won the 2001 Turing competition. – This is a “conversational” part of the robot brain, based on pattern-matching, parsing and black-board principles. – It is also a kind of “operating system” of the robot, which supervises other subroutines.
91. Behavior, Dialog and Learning • (2) Subroutines with logical data base and natural language parsing (CHAT). – This is the logical part of the brain used to find connections between places, timings and all kind of logical and relational reasonings, such as answering questions about Japanese geography.
92. Behavior, Dialog and Learning • (3) Use of generalization and analogy in dialog on many levels. – Random and intentional linking of spoken language, sound effects and facial gestures. – Use of Constructive Induction approach to help generalization, analogy reasoning and probabilistic generations in verbal and non-verbal dialog, like learning when to smile or turn the head off the partner.
93. Behavior, Dialog and Learning • (4) Model of the robot, model of the user, scenario of the situation, history of the dialog, all used in the conversation. • (5) Use of word spotting in speech recognition rather than single word or continuous speech recognition. • • (6) Continuous speech recognition (Microsoft) • (7) Avoidance of “I do not know”, “I do not understand” answers from the robot. – Our robot will have always something to say, in the worst case, over-generalized, with not valid analogies or even nonsensical and random.
94. Questions for Homeworks, Quizzes and Projects (1) 1. What is Braitenberg Vehicle? 2. Explain simple Braitenberg vehicles that operate on continuous (analog), binary and multiple-valued data. 3. Explain a model of robot behavior based on a single Finite State Machine. 4. Explain a model of robot behavior based on several communicating probabilistic Finite State Machines. 5. Explain a natural language model for a robot based on probabilistic and deterministic state machines. 6. What is the internal state of the robot? 7. Explain the concept of a behavior editor for a humanoid robot. 8. Find on internet about Eliza, Alice and other 1 chatbots and explain how they can be integrated in a complete robotic theatre of humanoid robots. 9. What is a single application of Machine Learning in robotics. 10. Think about possible applications of Machine Learning in humanoid robot theatre. 11. Draw a state diagram of FSM controlling a Braitenberg Vehicle type of robot with three inputs that can be switched from Aggressive to Shy behavior. 12. Draw a state diagram of a humanoid simple robot that behaves similarly to Aggressive and Shy Braitenberg Vehicles.
95. Questions (2) 1. What is the use of Avatar in robot design? 2. Give example of Multiple-Valued logic in Machine Learning applied to a robot. 3. Invent a humanoid robot head that would behave similarly to any Braitenberg Vehicle. Draw figure and State Machine for servos. 4. Explain simple Braitenberg vehicles that operate on fuzzy signals using Post Literals and MIN, MAX operators. 5. Write a software program that would use natural language and probabilistic state machine, similarly to examples above. 6. Design a robot FSM with three states : Happy, Unhappy and Neutral that would behave accordingly to one of the 1 three above moods. 7. Give concepts for a humanoid robot head with moveable eyes, eyebrows, ears, lips, cheeks and jaw. How to build it mechanically. Look critically at the pictures in this set of slides. 8. How to control all servos of these head parts – give ideas and explain how to relate them to facial gestures. 9. Think how to use Boolean Minimization to find generalized answers in question-answering. 10. Apply Braitenberg Vehicle ideas to Jimmy robot. 11. Apply Braitenberg Vehicle ideas to the robot from your project.