In 2017, a poker bot called Libratus made headlines when it roundly defeated four top human players at no-limit Texas Hold ‘Em. Now, Libratus’s technology is being adapted to take on opponents of a different kind—in service of the US military.
Libratus—Latin for balanced—was created by researchers from Carnegie Mellon University to test ideas for automated decision-making based on game theory. Early last year, the professor who led the project, Tuomas Sandholm, founded a startup called Strategy Robot to adapt his lab’s game-playing technology for government use, such as in wargames and simulations used to explore military strategy and planning. Late in August, public records show, the company received a two-year contract of up to $10 million with the US Army. It is described as “in support of” a Pentagon agency called the Defense Innovation Unit, created in 2015 to woo Silicon Valley and speed US military adoption of new technology.
Libratus’s defeat of poker pros in 2017 was seen as a milestone in AI because the card game has complex features lacking in the board games most prominently mastered by computers. In chess and Go, every piece is exposed for both players to see, making them what are called perfect information games. In poker, not all cards are visible, meaning that—as in many real-life scenarios—some information needed to calculate the true state of play is unknown.
Libratus was built on a technology called computational game theory. It won more than $1.8 million in play money from the poker champions by calculating how they might respond to its decisions. The software devised powerful betting strategies, and even showed the ability to bluff.
Sandholm says that approach can be applied to many other games, and also military simulations. Wargaming exercises typically test only small numbers of strategies for imagined opponents, even when run as computer simulations, he says. “That opens yourself up to a lot of exploitation, because the real adversary may not play according to your assumptions,” Sandholm says.
Sandholm declines to discuss specifics of Strategy Robot’s projects, which include at least one other government contract. He says it can tackle simulations that involve making decisions in a simulated physical space, such as where to place military units. The Defense Innovation Unit declined to comment on the project, and the Army did not respond to requests for comment.
Libratus’s poker technique suggests Strategy Robot might deliver military personnel some surprising recommendations. Pro players who took on the bot found that it flipped unnervingly between tame and hyper-aggressive tactics, all the while relentlessly notching up wins as it calculated paths to victory. “It’s weird because it doesn’t seem that it overwhelms you, but then you look at the score and you realize what’s happened,” Sandholm says.
Greg Allen, an adjunct fellow at think tank the Center for a New American Security, says the type of technology that powered Libratus could make wargaming and simulation exercises more useful. “It’s still far from real but it’s a better proxy for the real world,” he says. All the same, the results will likely remain just one component of strategy planning and research, he says, because the world is much more complex and messy than the scenarios even the best AI technology can master.
Strategy Robot isn’t the Pentagon’s only new foray into AI-enhanced game theory. Its research agency DARPA is starting a program to explore how the technology can be applied to military decision-making. Michael Wellman, a professor at the University of Michigan, says his group is working on applying computational game theory to cybersecurity under that program. He says Libratus can be seen as a sign that the technology is maturing. “It really is time to try this in some more real domains,” he says. “The breakthrough in poker was just so striking and things are going quickly with other games.”
In addition to Strategy Robot, Sandholm has founded a second startup called Strategic Machine, which is deploying his game-solving techniques in commercial settings, such as electricity markets, sports, and making computer-controlled players in videogames wilier adversaries. Both companies are bootstrapped, have roughly six employees, and are profitable, Sandholm says.
Back in his CMU computer lab, Sandholm is also thinking about how to make his technology more portable. When Libratus took on the poker pros it ran on the Bridges supercomputer at the federally funded Pittsburgh Supercomputing Center; Sandholm says his startups have also used supercomputers. At NeurIPS, the world’s largest AI conference last month, he and his collaborator on Libratus, Noam Brown, presented a paper on a less powerful but more compact poker bot called Modicum that can run on a single server. “In some applications you need it to be miniaturized, if it’s on board something,” Sandholm says. “Some platforms can’t carry big computers.”
The Pentagon is pushing to make broader use of AI technology. In 2017, then-Defense Secretary James Mattis lamented that his department lagged behind technology companies in the adoption of technologies like machine learning. That same year, the Pentagon started a program called Project Maven intended to employ commercially available AI techniques on US missions. Its initial project used machine learning to flag objects in drone surveillance video, with help from AI-savvy startups and large companies—including Google.
Other nations, too, are exploring military uses of AI. Russian President Vladimir Putin has said that whoever leads in AI “will become the ruler of the world.” Military applications feature prominently in China’s national AI strategy. In 2017, China’s National Defense University hosted a national war-gaming contest in which human teams took on an AI system.
The growing military interest in AI unsettles some technologists advancing the underlying technology. Some of Google’s AI researchers joined the thousands of employees who protested against the company’s work on Project Maven.
Sandholm believes concerns about US military use of AI are overblown. The technology is important to help the Pentagon keep the US safe and improve operational efficiency, he says. “I think AI’s going to make the world a much safer place,” Sandholm says.