Researchers find AI programs learn thousands of times faster compared to brute-force trial and error methods
When it comes to mastering video games, artificial intelligence programs can benefit greatly from a technique recommended to human users of new appliances but often ignored: Read the manual.
Researchers at Carnegie Mellon University led by Yue Wu put their AI program through the paces of learning how to play an old Atari video game. These decades-old games are often used to train AIs because they involve relatively simple visual inputs and basic controls, but are still somewhat difficult to master, as any GenXer who spent endless hours playing Pitfall can attest.
The researchers at Carngie Mellon chose Atari Skiing, since it involves a long wait between action and reward, making it particularly difficult for an AI to hit on a winning strategy. The goal of the game is to guide your skier down a slope, avoiding obstacles and steering between pairs of poles on the way to the bottom.
Previous AIs had relied on a kind of brute-force trial-and-error method, playing billions of games one frame at a time over a timespan that would have taken a human 85 years of play. Wu and his team began by giving their AI an instruction manual, either the original one packaged with the game, or a write-up from Wikipedia.
-
‘I am not a human — but I am trying’: An interview with an AI chatbot
-
Will Chat GPT take my job? And other burning AI questions
Because instruction manuals can be wordy and repetitive — one reason many humans avoid them — the team put the information through two other AIs first: an extractive module, to winnow out the basic rules; and a reasoning module to make sense of them.
Their AI was then able to master the game some 6,000 times faster than other methods — five days rather than 85 years of playing time. “To our knowledge, our work is the first to demonstrate the capability to improve Reinforcement Learning performance in an end- to-end setting using real-world manuals designed for human readers,” they wrote. (While the researchers’ AI did not quite beat the scores of other AIs, it was orders of magnitude better at efficiency.)
Ironically, newer video games often dispense with instruction manuals, relying on in-game tutorials and practice, or knowledge of earlier games, to get humans up to speed. So while an AI may be able to beat your high score at Atari Skiing or Pitfall, you’re like to outrank it at Call of Duty or Elder Scrolls — for now, at least.
