SAN FRANCISCO:T hat’s how much virtual computing time it took researchers at OpenAI, the non-profit artificial intelligence lab funded by Elon Musk and others, to train its disembodied hand. The team paid Google $3,500 to run its software on thousands of computers simultaneously, crunching the actual time to 48 hours. After training the robot in a virtual environment, the team put it to a test in the real world.
Ken Goldberg, a University of California, Berkeley robotics professor who isn’t affiliated with the project, said OpenAI’s achievement is a big deal because it demonstrates how robots trained in a virtual environment can operate in the real world. His lab is trying something similar with a robot called Dex-Net, though its hand is simpler and the objects it manipulates are more complex.
“The key is the idea that you can make so much progress in simulation,” he said. “This is a plausible path forward, when doing physical experiments is very hard.”
Dactyl’s real-world fingers are tracked by infrared dots and cameras. In training, every simulated movement that brought the cube closer to the goal gave Dactyl a small reward. Dropping the cube caused it to feel a penalty 20 times as big.
The process is called reinforcement learning. The robot software repeats the attempts millions of times in a simulated environment, trying over and over to get the highest reward. OpenAI used roughly the same algorithm it used to beat human players in a video game, “Dota 2.”