AlphaStar is a computer program by DeepMind that plays the video game StarCraft II. It was unveiled to the public by name in January 2019. In a significant milestone for artificial intelligence, AlphaStar attained Grandmaster status in August 2019.
Background
Games created for humans are considered to have external validity as benchmarks of progress in artificial intelligence. IBM's chess-playing Deep Blue (1997) and DeepMind's AlphaGo (2016) were considered major milestones; some argue that StarCraft would also be a major milestone, due to StarCraft's "real-time play, partial observability, no single dominant strategy, complex rules that make it hard to build a fast forward model, and a particularly large and varied action space." Though difficult, StarCraft may still be tractable with current technology because "its rules are known and the world is discrete with only a few types of objects".
StarCraft II is a popular fast-paced online real-time strategy game by Blizzard Entertainment.
History
DeepMind Technologies was founded in the UK in 2010. As early as 2011, founder Demis Hassabis called StarCraft "the next step up" after games like Go. DeepMind became a Google subsidiary in 2014, after demonstrating self-learning bots with superhuman ability at a variety of Atari 2600 games. In February 2015, computer scientist Zachary Mason predicted Deepmind's research "leads to StarCraft in five or ten years". In March 2016, following AlphaGo's victory over Lee Sedol, a world champion Go player, Hassabis publicly mulled building an AI for StarCraft, citing it as a strategic game with incomplete information where (unlike Go) much of the "board" is invisible. A formal collaboration was announced at BlizzCon in November 2016, alongside a plan to release an open development environment for bots in Q1 of 2017.
By 2017, DeepMind was experimenting with feeding StarCraft data into its software. In August 2017, DeepMind and Blizzard released development tools to assist in bot development, as well as data from 65,000 past games. At the time, computer scientist and StarCraft tournament manager David Churchill guessed it would take five years for a bot to beat a human, but made the caveat that AlphaGo had beaten expectations. In Wired, tech journalist Tom Simonite stated, "No one expects the robot to win anytime soon. But when it does, it will be a far greater achievement than DeepMind's conquest of Go."
On 19 December 2018, DeepMind's bot defeated "a top professional player", Grzegorz "MaNa" Komincz, 5-0. DeepMind announced the bot, named "AlphaStar", on 24 January 2019. A journalist at Ars Technica and others argued that AlphaStar still had unfair advantages: "AlphaStar has the ability to make its clicks with surgical precision using an API, whereas human players are constrained by the mechanical limits of computer mice". AlphaStar also had a global view rather than being limited by the in-game camera. Furthermore, while there was a cap on the number of actions over a five-second window, AlphaStar was free to allocate its action quota unevenly across the window in order to launch superhuman bursts of activity at critical moments. DeepMind quickly retrained AlphaStar under more realistic constraints, and then lost a rematch with Komincz. Starting in July 2019, the new, constrained version of AlphaStar anonymously competed against players who "opted in" on the public 1v1 European multiplayer ladder. By the end of August 2019, AlphaStar had attained "grandmaster level", ranking among the top 0.2 percent of human players.
Algorithms
Unlike AlphaZero, AlphaStar initially learns to imitate the moves of the best players in its database of human vs. human games; this step is necessary to solve what DeepMind's Dave Silver calls "the exploration problem": discovering new strategies would otherwise be like finding a "needle in a haystack". Agents then play each other and deploy reinforcement learning. These main agents also learn by playing against suboptimal "exploiter agents" whose purpose is to expose weaknesses in the main agents.
Reactions
After his 5-0 defeat in December 2018, Komincz stated "I wasn't expecting the AI to be that good".
Stuart Russell assessed that AlphaStar's 2018 victory required "a fair amount of problem-specific effort" and that general-purpose methods were "not quite ready for StarCraft".