A new AI agent developed by NVIDIA Research has successfully trained a robotic hand to perform intricate pen-spinning tricks, achieving the same level of skill as a human. The AI agent, called Eureka, utilizes reward algorithms to train robots in various tasks autonomously.

Eureka has taught robots to perform nearly 30 tasks with expertise, including opening drawers and cabinets, tossing and catching balls, and manipulating scissors. The research, which includes a paper and AI algorithms, is available for developers to experiment with using NVIDIA Isaac Gym, a physics simulation reference application for reinforcement learning research. Eureka is powered by the GPT-4 large language model.

One of the challenges in reinforcement learning is reward design, which often involves a trial-and-error process. Eureka addresses this challenge by integrating generative and reinforcement learning methods, paving the way for new algorithms that can solve complex tasks. The generated reward programs by Eureka outperform human-written ones on over 80% of tasks, resulting in an average performance improvement of more than 50% for the robots.

Eureka leverages the GPT-4 large language model and generative AI to write software code that rewards robots for reinforcement learning, without the need for task-specific prompts or predefined reward templates. It also incorporates human feedback to modify its rewards and align them more accurately with a developer’s vision.

The AI agent relies on GPU-accelerated simulation in Isaac Gym to efficiently evaluate large batches of reward candidates, enhancing the training process. Eureka continuously improves itself by analyzing key statistics from training results and instructing the language model to enhance its generation of reward functions. This self-improvement process allows Eureka to teach various types of robots, including quadruped, bipedal, quadrotor, dexterous hands, and cobot arms, to accomplish a broad range of tasks.

The research paper provides detailed evaluations of 20 Eureka-trained tasks, demonstrating the bots’ manipulation skills with open-source dexterity benchmarks. Visualizations generated using NVIDIA Omniverse showcase results from nine Isaac Gym environments.

Eureka’s combination of large language models and GPU-accelerated simulation technologies makes it a unique AI agent. NVIDIA researchers believe that Eureka will significantly contribute to dexterous robot control and provide a novel approach to creating physically realistic animations for artists.

This breakthrough work opens up new possibilities for developers and adds to NVIDIA Research’s previous advancements, such as Voyager, an AI agent capable of autonomously playing Minecraft and built with GPT-4.

Sources:
– NVIDIA Research