Summary: Equipping robots with social reasoning can make human-robot interaction smoother and more predictable, say MIT researchers.
Source: MIT
Robots can already deliver food on campus or make precise physical moves, but they struggle with simple social interactions that people rely on every day.
Researchers at MIT have designed a computational framework that adds social reasoning to robot behavior. In simulated environments, robots can infer another agent’s goals and choose actions that either help or hinder that agent, depending on their own objectives. The model lets machines learn and execute basic social behaviors—such as cooperation, obstruction, and joint assistance—by blending physical goals with social goals in a principled way.
The team demonstrated that the model produces interactions humans find realistic and easy to interpret. When people watched short clips of the simulated agents, their judgments about whether an interaction was helping or hindering generally matched the model’s classifications. This agreement suggests the approach captures human-relevant cues for social behavior.
Adding social skills to robots could improve human-robot interaction across many domains. In assisted living, for example, robots that understand when to offer help or when to prevent harm could create a more compassionate, effective care environment. Beyond engineering applications, the model provides a way to quantify social interactions that could aid psychological research on topics like autism or the behavioral impact of medications.
“Robots will live in our world soon enough, and they need to learn how to communicate with us on human terms,” says Boris Katz, principal research scientist and head of the InfoLab Group in MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and a member of the Center for Brains, Minds, and Machines (CBMM). “They need to understand when it is time to help and when it is time to prevent something from happening. This is early work, but it is one of the first serious attempts to model social interaction between humans and machines.”
The authors include co-lead Ravi Tejwani, co-lead Yen-Ling Kuo, Tianmin Shu, and senior author Andrei Barbu. Their findings will be presented at the Conference on Robot Learning in November.
A social simulation
To study social interaction, the researchers built a controlled two-dimensional simulation where agents pursue both physical and social goals. Physical goals are concrete objectives tied to the environment—such as navigating to a specific location. Social goals require an agent to infer what another agent is trying to accomplish and then act in support of or opposition to that inferred goal.
Their framework explicitly defines an agent’s physical objectives, its social objectives, and the relative weight it gives each. Agents receive rewards that reflect progress toward their goals. An agent that aims to help another aligns its reward with the other agent’s reward; one that aims to hinder inverts that reward. A planner algorithm continuously updates these rewards and selects actions that achieve a mixture of physical and social aims.
“We opened a new mathematical framework for modeling social interaction between agents,” Tejwani explains. “If you see another agent heading to location X, you can choose actions that make reaching X easier or harder. Our formulation specifies what social interactions mean mathematically, while the planner discovers how to carry them out.”
Balancing physical and social objectives produces more realistic robot behavior. Humans who help one another generally limit how far they will go; a model that allows unbounded helping would be unrealistic. The framework captures such trade-offs so agents act plausibly rather than naively.
Using this framework, the team defined three levels of agent sophistication. A level 0 agent pursues only physical goals and does not reason about others. A level 1 agent pursues physical and social goals but assumes other agents have only physical goals—it can therefore help or hinder based on perceived physical goals of others. A level 2 agent models that other agents are themselves social reasoners, enabling more sophisticated behaviors like coordinated assistance.
Evaluating the model
To test whether the simulated interactions matched human perceptions, the researchers generated 98 scenarios featuring level 0, 1, and 2 agents. Twelve human participants watched 196 video clips and judged the agents’ physical and social goals. In most cases, human judgments aligned with the model’s inferences about whether interactions were helping, hindering, or neutral.
“We want to go further than building computational models for robots,” says Andrei Barbu. “We also want to understand what visual and behavioral features humans use to interpret social interactions. If we can measure those features, we might develop objective tests of social-recognition ability or even training methods to improve that ability.”
Toward greater sophistication
Next steps include extending the simulation to three-dimensional agents and richer environments that allow object manipulation and a wider variety of social acts. The team also plans to model environments where actions may fail, and to replace the current planner with a neural-network-based planner that learns from experience and runs faster. Finally, they aim to run experiments to identify which perceptual cues humans rely on when judging whether agents are engaging socially.
“Ideally, we will establish a benchmark for social interaction recognition that the research community can use to accelerate progress,” Barbu adds.
Tomer Ullman, an assistant professor of psychology at Harvard University who was not involved in the work, praises the approach: “Even infants recognize helping and hindering. Machines lack that kind of flexible social reasoning, and models that let agents reason about others’ rewards and plan socially are an important step forward.”
Funding: This research was supported by the Center for Brains, Minds, and Machines; the National Science Foundation; the MIT CSAIL Systems that Learn Initiative; the MIT-IBM Watson AI Lab; the DARPA Artificial Social Intelligence for Successful Teams program; the U.S. Air Force Research Laboratory; the U.S. Air Force Artificial Intelligence Accelerator; and the Office of Naval Research.
About this robotics research news
Author: Adam Zewe
Source: MIT
Contact: Adam Zewe – MIT
Image: The image is in the public domain
Original Research: The full, open access research paper (pdf) is available for download from the conference proceedings.