Berkeley Startup to Train Robots Like Puppets

Estimated reading time: 3 minutes

Robots today must be programmed by writing computer code, but imagine donning a VR headset and virtually guiding a robot through a task, like you would move the arms of a puppet, and then letting the robot take it from there.

UC Berkeley’s robot BRETT was a breakthrough in deep reinforcement learning two years ago. Combining those skills with VR technology has made BRETT even better, allowing people to train BRETT and other industrial robots like a puppeteer. Video by Roxanne Makasdjian and Stephen McNally, with footage by Tianhao Zhang, UC Berkeley.

That’s the vision of Pieter Abbeel, a professor of electrical engineering and computer science at the University of California, Berkeley, and his students, Peter Chen, Rocky Duan and Tianhao Zhang, who have launched a startup, Embodied Intelligence Inc., to use the latest techniques of deep reinforcement learning and artificial intelligence to make industrial robots easily teachable.

“Right now, if you want to set up a robot, you program that robot to do what you want it to do, which takes a lot of time and a lot of expertise,” said Abbeel, who is currently on leave to turn his vision into reality. “With our advances in machine learning, we can write a piece of software once — machine learning code that enables the robot to learn — and then when the robot needs to be equipped with a new skill, we simply provide new data.”

The “data” is training, much like you’d train a human worker, though with the added dimension of virtual reality. Using a VR headset without ever touching the robot, people can train a robot in a day, in contrast to the weeks to months typically required to write new computer code to reprogram a robot. The technique can work with robots currently in manufacturing plants and warehouses around the world.

“Commodity VR devices provide an easy way to control physical robots. Since the robot simply mimics the hand motion that’s tracked by VR, a person without any special training can make the robot do the right thing right from the beginning,” Chen said. “The robot will keep learning and after a while the robot says, ‘I got this, I can do this task on my own now.’”

In a paper posted online last month, Abbeel and his colleagues demonstrated the power of this type of imitation learning: Using a $1,000 VR headset and hand-tracking software, they trained a robot to coordinate its arms with its vision to learn new skills as complex as inserting a peg into a hole.

“It completely changes the turnaround time because the amount of data you need is relatively small,” Abbeel said. “You might only need a day of demonstrations from humans to have enough data for a robot to acquire the skill.”

“When we perform a task, we do not solve complex differential equations in our head. Instead, through interactions with the physical world, we acquire rich intuitions about how to move our body, which would be otherwise impossible to represent using computer code,” Duan said. “This is much like AlphaGo, which does not use any of the hard-coded strategies common in traditional approaches, but acquires its own intuitions and strategies through machine learning.”

AlphaGo is a computer program developed by Alphabet Inc. to play the ancient Chinese board game Go, which is considered more complicated for a computer than either checkers or chess. Using machine learning, AlphaGo earlier this year beat the world’s top-ranked Go player.

Abbeel, who is president and chief scientist of the startup, cofounded the company in September with three of his graduate students: Chen, now CEO; Duan, now CTO; and Zhang, now on the technical staff. Based in Emeryville, just south of Berkeley, it has already raised $7 million in seed funding.

Abbeel, Chen, Duan and Zhang have worked together for many years in the Berkeley AI Research lab. Abbeel, Chen and Duan also worked together at OpenAI, a non-profit company cofounded by Elon Musk, of Tesla and Space-X fame, and dedicated to building safe AI.

The idea behind the company came from the team’s observation that rapid advances in deep reinforcement learning and deep imitation learning over the past five years are not reflected in the industrial robots in use today to assemble cars and appliances or move stuff around warehouses.

“This is an amazing capability that we just developed here at UC Berkeley, and we decided we should put this into the world and empower companies still using techniques that are many years behind what is currently possible,” Abbeel said. “This will democratize access to robotic automation.”