AI bots just beat humans at the video game Dota 2

After recent Artificial Intelligence (AI) victories over top human players in the games of Go and poker, a new AI breakthrough involved a game that many people haven’t heard of, Dota 2.

Dota 2 is a multiplayer online battle arena (MOBA) video game in which two teams of five players compete to collectively destroy a large structure defended by the opposing team known as the “Ancient”, whilst defending their own. The game is controlled using standard real-time strategy controls, and is presented on a single map in a three-dimensional isometric perspective. Ten players each control one of the game’s 116 playable characters, known as “heroes”, with each having their own design, strengths, and weaknesses, the heroes are divided into two primary roles, known as the “carry” and “support”. Carries, which are also called “cores”, begin each match as weak and vulnerable, but are able to become more powerful later in the game, thus becoming able to “carry” their team to victory. Supports generally lack abilities that deal heavy damage, instead having ones with more functionality and utility that provide assistance for their carries, such as providing healing and other buffs. Players select their hero during a pre-game drafting phase, where they can also discuss potential strategies and hero matchups with their teammates. Heroes can not be switched mid-game, and once one is selected, they are removed from the drafting pool and become unavailable for all other players.

The game has been used in machine learning experiments, with artificial intelligence research company OpenAI curating a system that allows bots to learn how to play the game at a high skill level entirely through trail-and-error algorithms. In a system that OpenAI calls “reinforcement learning”, the bots learn over time by playing against itself hundreds a times a day for months, in which they are rewarded for actions such as killing an enemy and destroying towers.Demonstrations of the bots playing against professional players have occurred at a number of events, such as Dendi, a professional Ukrainian player of the game, losing to one of them in a live 1 vs 1 matchup at The International 2017.

A year later, the ability of the bots had increased to work together as a full team of five, known as the OpenAI Five, who then played and won against a team of semi-professional players in a demonstration game in August 2018. Shortly after, OpenAI Five then played two live games against more skilled players at The International 2018.

Although the bots lost both games, OpenAI considered it a successful venture, stating that playing against some of the best players in Dota 2 allowed them to analyze and adjust their algorithms for future games.

It’s also significant to AI researchers, especially those in companies such as Google, Facebook, Microsoft and IBM, which are investing millions of dollars in creating superhuman AI players for digital games. As AI becomes ever more important in our society, it could have wider implications for all of us because of what it demonstrates about computers’ ability to “think” strategically.

What was particularly remarkable about the Dota 2 victory, achieved by a bot was that its developers didn’t program it with deep understanding of game strategies. Instead, they used an approach known as deep reinforcement learning, where the computer starts with only rudimentary knowledge of game strategy.

By playing against itself millions of times, the AI learns to differentiate good move decisions (that lead to victory) from bad ones. The knowledge is stored in a huge data matrix containing millions of numbers, updated after every self-play game. These numbers encode what’s known as a “function”, the instructions that specify the AI’s learned strategy for every possible game situation. So after the AI researchers programmed the method for learning, the machine effectively taught itself how to make good move decisions.

The feat impressed Microsoft co-founder Bill Gates too.

I want to duo with an AI bot in Fortnite. I might actually win! 🙂