Studies have shown that humans find AI to be a frustrating teammate when playing cooperative games together and a challenge for “team intelligence”.
When it comes to games like chess and go, Artificial Intelligence (AI) programs far outperform the best players in the world. These “superhuman” AIs are unmatched competitors, but perhaps more difficult than competing with humans is to work with them. Can the same technology get along with people?
In a new study MIT Lincoln Laboratory researchers use advanced AI models that are well trained to play with teammates they have never met before, and how humans play collaborative card games fireworks. I looked for what I could do. In the single-blind experiment, participants played two sets of games. One used an AI agent as a teammate and the other used a rules-based agent, a bot manually programmed to play in a predefined way.
The results surprised the researchers. Not only did AI teammates perform worse than rule-based agents, humans have always hated playing with AI teammates. They found him to be unpredictable, unreliable, unreliable and felt negative even when the team gave a good score. An article detailing this study was accepted by the 2021 Neural Information Processing Systems (NeurIPS) Conference.
When playing the “Fireworks” cooperative card game, humans were dissatisfied and confused by the movements of their AI teammates. Credit: Bryan Mastergeorge
“This highlights the subtle difference between creating an AI that works objectively well and creating an AI that is subjectively reliable or prioritized,” the article co-wrote. Said Ross Allen, researcher in the artificial intelligence technology group. “They’re so close to each other that they can appear to be out of the sun, but this study showed that they’re actually two separate issues. We have to work to untangle them. “
Humans who dislike AI teammates may worry researchers who design this technology in collaboration with humans about real-world challenges such as missile protection and performing complex surgeries. This dynamic, called team intelligence, is the next frontier in AI research and uses a specific type of AI called reinforcement learning.
Reinforcement learning The AI is not informed of the action to be taken, but by experimenting with the scenario several times, it discovers which action provides the most digital “reward”. It is this technology that has created superhuman chess and go players. Unlike rule-based algorithms, these AIs are not programmed to follow “if / then” instructions. This is because the results of human tasks you plan to work on, such as driving a car, are too high to be coded.
“Reinforcement learning is a much more versatile way to develop AI. If you can train him to learn to play chess, that agent isn’t necessarily going to drive a car. But with the right data, you can use the same algorithm to train another officer to drive a car, ”says Allen. “Theoretically, the sky is the limit of what it can do.”
Bad advice, bad game
Today, researchers are testing the performance of reinforcement learning models developed for collaboration in the same way that chess has served as a benchmark for testing competitive AI for decades. I use fireworks to do this.
The fireworks game is similar to the multiplayer format of Solitaire. Players work together to stack cards of the same suit in order. However, players cannot see their cards, only cards held by their teammates. Each player has severe restrictions on what can be communicated to their teammates, allowing them to choose the best card in their hand and then crush it.
Lincoln Laboratory researchers did not develop the AI or the rules-based agent used in this experiment. Both agents are the best in their respective fields of fireworks performance. real, the AI model was previously paired with an AI teammate who never played with them, the team achieved the highest score ever in a fireworks display between two unknown AI agents .
“It was a big result,” says Allen. “I thought if these AIs that I had never met before could come together and play really well, I should be able to bring in people who know how to play AI as well. They are also doing very well. That’s why I thought the AI team could play well objectively, and I thought humans like it because, in general, if it works, they like things better. “
None of these expectations were met. Objectively, there was no statistical difference in scores between AI and rule-based agents. Subjectively, all 29 participants reported a clear preference for rule-based teammates in the survey. Participants were not told which agent and what game they were playing.
“One participant said he actually had a headache because he was stressed out by the AI agent’s bad game,” said Jaimepena, researcher and author of the AI Technology and Group article. Systems. “Another said rule-based agents thought they were ridiculous but achievable, but AI agents showed they understood the rules, but the movement was the look of the team. didn’t fit. For them, it gave a bad clue and played badly. “
This perception that AI is “playing badly” is linked to the surprising behavior previously observed in reinforcement learning tasks. For example, when DeepMind’s AlphaGo first beat one of the best Go players in the world in 2016, one of the most admired moves made by AlphaGo was move 37 in game 2 An Unusual Move That The Human commentators thought it was a mistake. Subsequent analysis revealed that this move was actually very well calculated and described as a “genius”.
Such moves can be praised when performed by an AI opponent, but less likely to be celebrated in a team setting. Researchers at the Lincoln Laboratory have discovered that weird or seemingly illogical moves are the worst criminals for breaking human trust in the AI teammates of these tightly coupled teams. Such movements not only reduce the player’s perception of how the player and their AI teammates work together, but also the amount of AI used, especially when the potential rewards are not immediately apparent. I also lowered what I wanted to do.
“There have been a lot of comments about giving up, like ‘I hate to handle this’,” said Hosea Siu, author of the article and researcher in the Autonomy and Control Systems Engineering group. Add.
Participants who rated themselves as fireworks experts, led by the majority of players in this survey, often abandoned AI players. Siu sees this as a concern for AI developers, as the main users of this technology are likely experts in the field.
“Let’s say you train a super intelligent AI guidance assistant for a missile defense scenario. You don’t give it to an intern. You give it to your expert on your ship who has been doing this for 25 years. Therefore, if there is a strong expert bias against it in the game scenario, it can manifest itself in actual operations, ”he adds.
Squeeze the human
The researchers say the AI used in this study was not developed for human taste. But that’s part of the problem – not much. Like most collaborative AI models, this model is designed to score as high as possible, and its success is measured by its objective performance.
If the researchers don’t focus on the question of subjective human tastes, “you can’t create the AI that humans really want to use,” says Allen. “It’s easier to work with AI which improves very sharp numbers. It is much more difficult to work on an AI that works in this favorite world of the humans of Musier. “
Solving this more difficult problem is the goal of the MeRLin (Mission-Ready Reinforcement Learning) project. The experiment was funded by the technical office of MIT Lincoln Laboratory in conjunction with the U.S. Air Force’s Artificial Intelligence Accelerator and MIT’s Electrical and Computer Engineering Division. Chemistry. This project investigates what keeps collaborative AI technology from leaping out of the play space and into more delicate reality.
Researchers believe that AI’s ability to explain its behavior builds trust. This will be the focus of their work next year.
“I can imagine we would repeat the experiment, but after the fact – and it’s not as easy as it looks – the humans said, ‘I don’t understand why you made that move. Did you do that? “If the AI can give some insight into what they think is going to happen based on their behavior, our guess is that humans ‘oh, weird thought, but I get it now.’ Trust it. Even if you don’t change basic AI decisions, the results will change completely, “Allen says.
Like post-game chats, this type of exchange often helps humans form friendships and cooperate as a team.
“It may also be a staff prejudice. Most AI teams don’t want to tackle these squeaky people and their limp issues, ”Siu adds with a laugh. “People want to do math and optimization. This is the basis, but it is not enough.
Mastering fireworks-like games between AI and humans has the potential to open up a world of possibilities for teaming up with intelligence in the future. But the technology can remain mechanical or human until researchers can bridge the gap between AI performance and human taste.
Reference: Ho Chit Siu, Jaime D. Pena, Kimberlee C. Chang, Edenna Chen, Yutai Zhou, Victor J. Lopez, Kyle Palko, RossE “Human AI Team Assessment for Rule-Based Agents Learned to Hanabi “”. Allen, accepts, 2021 Neural Information Processing Systems (NeurIPS) Conference..
Artificial Intelligence Is Smart, But Doesn’t Work Well With Others Source Link Artificial Intelligence Is Smart, But Doesn’t Work Well With Others