Nvidia Leaps into the Robot Future with GR00T

In a week when Nvidia announced collaborations on AI with nearly every tech company on earth – from Siemens and MiR to Teradyne and HP – we also saw the announcement of the AI robot project GR00T. One of Nvidia's new initiatives is Jetson Thor, a computer designed for humanoid robots based on the the company's Thor system-on-a-chip (SoC). Nvidia also introduced upgrades to the Isaac robotics platform. Advances include generative AI foundation models and tools for simulation and AI workflow infrastructure. For a more complete view of Nvidia initiatives announced at the company's annual GTC developers conference, check out Spencer Chin's Nvidia Bolsters Its AI Efforts at GTC.

Jensen Huang, founder and CEO of Nvidia, explained the thinking behind the company’s work on humanoid robotics. “Building foundation models for general humanoid robots is one of the most exciting problems to solve in AI today,” said Huang. “The enabling technologies are coming together for leading roboticists around the world to take giant leaps towards artificial general robotics.”

A Robot Created to Relate to Humans

Robots powered by GR00T, which stands for Generalist Robot 00 Technology, will be designed to understand natural language and emulate movements by observing human actions. The robot will be able to quickly learn coordination, dexterity and other skills in order to navigate, adapt and interact with the real world. In his GTC keynote, Huang demonstrated several such robots completing a variety of tasks (zee video above).

Jetson Thor was created as a new computing platform capable of performing complex tasks and interacting safely and naturally with people and machines. It has a modular architecture optimized for performance, power, and size.

Nvidia’s SoC includes a next-generation GPU based on the Nvidia Blackwell architecture with a transformer engine delivering 800 teraflops of 8-bit floating point AI performance to run multimodal generative AI models like GR00T. With an integrated functional safety processor, a high-performance CPU cluster and 100GB of ethernet bandwidth, it significantly simplifies design and integration efforts.

Bringing AI to Robot Companies

In its release about GR00T, Nvidia noted the company is building a comprehensive AI platform for leading humanoid robot companies such as 1X Technologies, Agility Robotics, Apptronik, Boston Dynamics, Figure AI, Fourier Intelligence, Sanctuary AI, Unitree Robotics, and XPENG Robotics, among others.

“We are at an inflection point in history, with human-centric robots like Digit poised to change labor forever. Modern AI will accelerate development, paving the way for robots like Digit to help people in all aspects of daily life,” said Jonathan Hurst, cofounder and chief robot officer at Agility Robotics. “We’re excited to partner with Nvidia to invest in the computing, simulation tools, machine learning environments and other necessary infrastructure to enable the dream of robots being a part of daily life.”

Major Updates to the Isaac Platform

According to Nvidia, the Isaac tools that GR00T utilizes are capable of creating new foundation models for any robot embodiment in any environment. Among these tools are Isaac Lab for reinforcement learning, and OSMO, a compute orchestration service.

The new Isaac Lab is a GPU-accelerated, lightweight, performance-optimized application built on Isaac Sim specifically for running thousands of parallel simulations for robot learning. To scale robot development workloads across heterogeneous compute, OSMO coordinates the data generation, model training and software/hardware-in-the-loop workflows across distributed environments.

Nvidia also announced Isaac Manipulator and Isaac Perceptor – a collection of robotics pretrained models, libraries and reference hardware.

Nvidia noted that Isaac Manipulator offers state-of-the-art dexterity and modular AI capabilities for robotic arms, with a robust collection of foundation models and GPU-accelerated libraries. It provides up to an 80x speedup in path planning and zero-shot perception increases efficiency and throughput, enabling developers to automate a greater number of new robotic tasks. Among early ecosystem partners are Yaskawa, Universal Robots, a Teradyne company, PickNik Robotics, Solomon, READY Robotics, and Franka Robotics.

Isaac Perceptor provides multi-camera, 3D surround-vision capabilities, which are increasingly being used in autonomous mobile robots adopted in manufacturing and fulfillment operations to improve efficiency and worker safety as well as reduce error rates and costs. Early adopters include ArcBest, BYD and KION Group as they aim to achieve new levels of autonomy in material handling operations.