Robot communication: It's more than just talk
C-3PO鈥檚 fluency in聽聽in 鈥淪tar Wars鈥� set a high bar for human-robot interaction, and the field has been struggling to catch up ever since.
They started in the factories, taking over physically demanding and repetitive tasks. Now robots are moving into聽,听, even the聽, and experts don鈥檛 expect their expansion into human spaces to slow down anytime soon.
鈥淓ven 10 years ago, the primary use of the robots was in the dangerous, dirty, and dull work,鈥� says Julie Shah, an engineering professor at the Massachusetts Institute of Technology in Cambridge, Mass. 鈥淵ou鈥檇 deploy them to operate remotely from people, but [now] robots are integrating into all aspects of our lives relatively quickly.鈥�
Freed from their isolated industrial cages, robots navigating the human world can pose hazards to聽聽and others, so researchers are seeking ways to prepare for a future where people and robots can work safely together.
While they wouldn鈥檛 have made his official list, C3PO鈥檚 most important forms of communication may have been nonverbal. We absorb a staggering amount of information visually, from gestures and facial expressions to traffic lights and turn signals, and good design can take advantage of that skill to let humans meet robots halfway.
Signaling to others
Holly Yanco,听computer聽science professor at the University of Massachusetts, Lowell, suggests early measures could be as simple as equipping robots with universal icons.聽
鈥淚 may not need to know everything that the robot is doing, but I need to know that this space is safe for me to walk into,鈥� explains Professor Yanco.
A survey conducted by her graduate student, Daniel Brooks, found聽聽such as checks or question marks sufficient to communicate a robot鈥檚 status to untrained bystanders.聽Think of the聽聽from 鈥淲all-E.鈥�
Such iconography still depends on culture, Yanco is quick to point out. Another path involves giving robots something all humans have experience reading.
Rethink Robotics takes this approach with its dual-armed Baxter, which features a cartoon face displayed on a swiveling tablet. Thanks to Baxter鈥檚 animated eyes, human coworkers can know at a glance where its attention lies and which arm it may be moving next.
People watching
Even if robots become open books, that鈥檚 only half of the equation. Dr. Shah heads MIT鈥檚 Interactive Robotics Group, a lab focused on giving robots mental and sensory flexibility to complement their physical prowess.
They aim to build robotic systems that can work alongside, and even integrate with, human teams. And that means robots that learn from observation, predict teammate actions, and adjust their behavior accordingly, much like a person would. 鈥淚 don鈥檛 think this is a very futuristic idea anymore,鈥� Shah says.
In fact, the group tested just such a system last year. After an 鈥渁pprenticeship鈥� spent watching nurses and doctors, a robotic decision support system succeeded in making patient care suggestions that nurse participants in controlled experiments accepted 90 percent of the time. The study culminated in a聽聽of a Boston hospital, where the system gathered inpatient data from handwriting on a whiteboard and offered real time advice.
鈥淭hat was the first time anybody has been able to demonstrate a system learning so efficiently with so few demonstrations in a real world setting,鈥� says Shah. 鈥淚t can be done.鈥�
Still, even the most mentally dextrous teammate will sink a project if they can鈥檛 keep out of the way. 鈥淲hen you start working in a confined space, an elbow-to-elbow space, body posture and motion signals become very important. People rely on them,鈥� says Shah.
Her team also harnesses machine learning and biophysical modeling to help robots read human body language, and predict where a teammate will move next. For example, tracking a person鈥檚 walking speed and head direction reveals聽聽about two steps early,听information聽we humans only become aware of when a miscalculation ends in the 鈥渉allway dance.鈥�
鈥淐learly we all use these cues聽everyday聽but we don鈥檛 think about it,鈥� says Shah. 鈥淛ust a quarter of a second or half a second of an arm reaching movement ... with just the slightest聽motion聽of their elbow or their shoulder, we can train a machine learning technique to predict with 75 percent accuracy where they鈥檙e going to reach.鈥�
Reading minds?
While Yanco and Shah help catch robots up to people鈥檚 signaling and interpreting abilities, other researchers see no reason to limit robots to human senses. Another system developed by MIT鈥檚 Computer Science and Artificial Intelligence Laboratory (CSAIL) can read minds. Or at least one very specific thought.
While researching brain-to-computer interfaces with monkeys, Boston University Prof. Frank Guenther was struck by how one particular signal came through unusually clearly against the cacophonous background of neural activity. When the monkeys noticed that the computer under their control had made a mistake, a cursor moving right when they had thought 鈥済o left,鈥� for example, the system registered a so-called 鈥渆rror potential.鈥�
The signal was strong enough to be detected via an electrical cap and featured a relatively similar shape from person to person. A collaboration between CSAIL and Dr. Guenther鈥檚 lab succeeded in designing a system that let a Baxter robot sort paint cans and wire spools into two buckets by 鈥渓istening鈥� for error potentials, randomly guessing at first and then self-correcting if it noticed the user thinking it made a mistake.
At around 85 percent accuracy, the system isn鈥檛 ready for the factory floor, but Guenther expects eventual applications such as a human overseeing a self-driving car or a supervisor monitoring manufacturing machines.
鈥淲e鈥檙e capitalizing on the fact that the human brain already has a tremendous amount of circuitry built for understanding things and if it sees a mistake, that mistake can be at a pretty high level and still generate a signal,鈥� he says.
And there鈥檚 no reason to expect machines to stop at error potentials. Guenther can imagine a future where smartphone cameras measure pupil dilation and cases measure skin resistance (much like today鈥檚 lie detectors) to read the user鈥檚 emotions and respond more empathically.
A functional C-3PO may still be a long way off, but聽Yanco agrees that we鈥檝e just begun to see what鈥檚 possible when robots and humans join forces. 鈥淲e鈥檙e still in the very early days,鈥� says Yanco. 鈥淚 think there鈥檚 still a lot of exploration to go.鈥�
