Outmaneuvering Gran Turismo Drivers with AI

Peter Wurman ’87 was just a kid growing up in Wisconsin when the first personal computers hit the market. At 13, he used his bar mitzvah money to buy an Apple II. “The only thing you could do with it was program it, so one of my hobbies was learning to program on the Apple,” Wurman says. 

In the long term, this hobby has paid off handsomely for Wurman. In May 2022, he was inducted into the National Inventors Hall of Fame for his pioneering work on Kiva, a robotic system that revolutionized warehouse fulfillment. And he currently serves as director of Sony AI America, which last year unveiled an artificial intelligence (AI) agent capable of beating the best human players at Gran Turismo Sport, a highly realistic driving video game.

Initially, Wurman was slow to embrace the idea of a career in computer science. “When I went to MIT, I was intimidated by computer science, and Unix, and all the big computers they had there, so I didn’t major in that,” he says. “I was really interested in design, so I majored in mechanical engineering.”

After graduating from MIT, Wurman earned a master’s degree in mechanical engineering at the University of Michigan. He also got a job on campus as a programmer, which helped him realize he could succeed in computer science. So, he decided to get his next degree—a doctorate—in computer science. “I was probably the most naïve PhD student to ever start at Michigan. There were basic computer science topics that I was learning for the first time in graduate courses,” Wurman says. 

Wurman went on to join the computer science faculty at North Carolina State University. Just a few years later, though, his old MIT roommate, Mick Mountz ’87, reached out for help with an idea he had for optimizing warehouses by using robots to bring merchandise to packers. “The idea was brilliant,” Wurman says. “But Mick didn’t know how to build robots, and he didn’t know how to write the software.”

Fortunately, Wurman could write software, and a third cofounder, Raffaello D’Andrea, could build robots. In 2004, while on sabbatical, Wurman cofounded Kiva Systems, and in 2007 he left NC State behind and moved to Boston permanently. The business went on to become a resounding success: Amazon acquired the company in 2012 for $775 million.

Wurman worked at Amazon for a few years, then in 2016 joined a startup, Cogitai, as vice president of engineering. That business, which focused on using reinforcement learning to enhance AI agents (the term used for computer programs that perceive and interact with environments), was acquired by Sony in 2019. Wurman joined Sony AI, and soon his team had developed GT Sophy, the AI agent that successfully used reinforcement learning to become the best player of Gran Turismo Sport in the world in 2021. 

Reinforcement learning, Wurman explains, basically works like training a dog. The computer programming provides “treats” for good behaviors and penalties for mistakes in the form of positive and negative feedback. Using this feedback, GT Sophy was able to learn to navigate the Gran Turismo racecourse essentially by trial and error. 

“With reinforcement learning, we don’t have to know the answer in order to program the computer,” Wurman says. “We let the agent learn how to drive by itself.”

This technique could one day be applied to enable AI agents to improve a broad range of complex systems such as robotics and industrial automation, Wurman says. But for now, he is focused on the fun stuff—working to incorporate GT Sophy as a player in the commercially available Gran Turismo Sport game and developing new AI agents for other games. 

“Video games are a huge industry, and entertainment is a huge industry,” he says. “Being able to apply agents to commercial games makes them more engaging and fun to play.”

Peter Wurman is a mentor with the Martin Trust Center for MIT Entrepreneurship.

Images: Courtesy of Sony AI