Our latest advances in robotic dexterity

Two new AI systems, ALOHA Unleashed and DemoStart, help robots perform complex tasks that require skillful movements

People perform many tasks every day, such as tying shoelaces or tightening a screw. But it is incredibly difficult for robots to properly learn these highly skilled tasks. To make robots more useful in people's lives, they must improve contact with physical objects in dynamic environments.

Today we introduce two new articles that showcase our latest advances in artificial intelligence (AI) in robot dexterity research: ALOHA Unleashed, which helps robots perform complex and novel two-arm manipulation tasks; and DemoStart, which uses simulations to improve the real-world performance of a multi-fingered robotic hand.

By helping robots learn from human demonstrations and put images into action, these systems pave the way for robots that can perform a variety of helpful tasks.

Improving imitation learning with two robotic arms

Until now, most advanced AI robots could only pick up and place objects with a single arm. In our new paper we present ALOHA Unleashed, which achieves a high level of skill in two-arm manipulation. Using this new method, our robot learned to tie a shoelace, hang a shirt, repair another robot, shift into gear, and even clean a kitchen.

The ALOHA Unleashed methodology builds on our ALOHA 2 platform, which was based on Stanford University's original ALOHA (a low-cost, open-source hardware system for bimanual teleoperation).

ALOHA 2 is significantly more dexterous than previous systems because it has two hands that can be easily teleoperated for training and data collection purposes, and it allows robots to learn how to perform new tasks with fewer demonstrations.

We have also improved the ergonomics of the robot hardware and improved the learning process in our latest system. First, we collected demonstration data by remotely controlling the robot's behavior and performing difficult tasks such as tying shoelaces and hanging t-shirts. Next, we applied a diffusion method that predicts robot actions from random noise, similar to how our Imagen model generates images. This helps the robot learn from the data so that it can perform the same tasks independently.

Learning robot behavior through a few simulated demonstrations

Controlling a dexterous robotic hand is a complex task that becomes even more complex with each additional finger, joint and sensor. In another new article, we introduce DemoStart, which uses a reinforcement learning algorithm to help robots learn skillful behavior in simulation. These learned behaviors are particularly useful for complex embodiments, such as: B. Hands with multiple fingers.

DemoStart initially learns from simple states and over time begins to learn from more difficult states until it masters a task as well as possible. 100 times fewer simulated demonstrations are required to learn how to solve a task in simulation than is typically required when learning from real-world examples for the same purpose.

The robot achieved a success rate of over 98% on a number of different tasks in the simulation, including realigning cubes with a specific color representation, tightening nuts and bolts, and tidying up tools. In the real setup, it achieved a 97% success rate in realigning and lifting the cube and 64% in a male-female insertion task that required high finger coordination and precision.

We developed DemoStart using MuJoCo, our open source physics simulator. After mastering a series of tasks in simulation and using standard techniques to reduce the gap between simulation and reality, such as: B. domain randomization, our approach was able to transfer near zero-shot to the physical world.

Robotic learning in simulation can reduce the cost and time required to conduct actual physics experiments. However, these simulations are difficult to design and do not always translate successfully into real-world performance. By combining reinforcement learning with learning from some demonstrations, DemoStart's progressive learning automatically generates a curriculum that bridges the gap between simulation and reality, making it easier to transfer knowledge from a simulation to a physical robot and the costs required to do so and reduce time conducting physical experiments.

To enable more advanced robotic learning through intensive experimentation, we tested this new approach on a three-fingered robotic hand called DEX-EE, developed in collaboration with Shadow Robot.

The future of robotic dexterity

Robotics is a unique area of ​​AI research that shows how well our approaches work in the real world. For example, a large language model could tell you how to tighten a bolt or tie your shoes, but even if it were embodied in a robot, it would not be able to perform these tasks itself.

One day, AI robots will help people with all sorts of tasks at home, at work, and more. Skills research, including the efficient and general learning approaches we have described today, will help make this future possible.

We still have a long way to go before robots can grasp and manipulate objects with the ease and precision of humans, but we are making significant progress and every breakthrough innovation is another step in the right direction.