‘It’s Like Playing Chess Blind’: Competition Tests AI Decision-Making Under Uncertainty

Last fall, more than 60 teams from around the world signed up for a competition developed by the Johns Hopkins Applied Physics Laboratory (APL), in Laurel, Maryland, to advance machine decision-making under uncertainty. Registrants hailed from top universities such as Carnegie Mellon and commercial leaders in artificial intelligence like Google and Microsoft.

In December, a group of APL staff members traveled to Vancouver to share the outcomes of the competition at the 2019 Conference on Neural Information Processing Systems (NeurIPS) — considered the top AI conference in the world — and the Laboratory recently launched the 2020 Reconnaissance Blind Chess (RBC) league for the competition to support ongoing research, this time all online.

RBC, invented by APL researchers, is a powerful twist on chess that incorporates uncertainty and sensing: players can’t see the positions of their opponent’s pieces, except through private sensing actions.

“It’s like playing chess blind, where each player gets to sense a 3-by-3 square of the board each turn,” explained APL’s Ryan Gardner, a computer scientist and one of the game organizers.

The Intelligent Systems Center has hosted several RBC tournaments — starting with an inaugural internal challenge in 2017. “Last year we pitched it to NeurIPS, which draws over 10,000 people a year. The conference is extremely competitive — the fact that our competition was accepted is an affirmation of the research value of the game,” Gardner said.

The RBC NeurIPS competition provided new insight into the strengths and limitations of many popular AI techniques — such as those that have achieved superhuman performance in games like Go and poker — in addressing the particular challenges in RBC.

Gino Perrotta, then a postdoctoral researcher at George Washington University, won the competition by creating an algorithm that tracks and reasons over all possible board states. Perrotta has since joined APL as a roboticist in the Laboratory’s Intelligent Systems Center and is working on the next generation of autonomous aerial vehicles.

“It was fascinating to try and create an algorithm that played the game like we did,” he said. “I know there is much more to be learned from exploring the game, and significant room for improvement among even the best bots.”

Even as the Lab expands its AI research portfolio, the ISC team will continue to foster a global research community around RBC and other challenge problems geared toward using AI to address critical challenges in real-world applications.

RBC organizers now have a dynamic online leaderboard with cash prizes ($1,000 for first place and $500 for second place) offered to the top-ranked bots at the end of August 2020.

Additional information is available at https://rbc.jhuapl.edu.