When Jensen Huang, CEO of Nvidia, says, “Building foundation models for general humanoid robots is one of the most exciting problems to solve in AI today,” it’s hard to disagree. Humanoid form factors are attracting a lot of investment money and skepticism, making them one of the biggest topics of discussion in robotics right now.
Without a doubt, Nvidia is interested in getting its hands on some. With projects like Jetson and Isaac, the chip behemoth has lately established itself as an important player in the advancement of robotics, and it has grown into perhaps the most significant AI hardware business. Project GR00T, unveiled last week at the company’s annual GTC developer conference, may or may not be a reference to Marvel’s evil talking space tree, but it certainly puts the company in the humanoid race.
The chip manufacturer calls the new system “a general-purpose foundation model for humanoid robots.” An artificial intelligence platform is being constructed by the company in order to accommodate the recent influx of companies that have entered the sector. These companies include Sanctuary AI, 1X Technologies, Unitree Robotics, Agility Robotics, Boston Dynamics, Apptronik, Fourier Intelligence, Figure AI, and XPENG Robotics. An AI platform is being constructed by the company. That pretty well covers all of the major manufacturers of humanoid robots right now, with a couple of significant exceptions, like Tesla.
With a comment from co-founder and Chief Robotics Officer Jonathan Hurst: “We are at an inflection point in history, with human-centric robots like Digit poised to change labor forever,” Agility receives further attention in the release. He also mentioned that with the rapid advancement of modern AI, robots like Digit will soon be able to assist humans with every part of their daily lives. He said they are thrilled to be working with NVIDIA to fund the infrastructure—including computers, machine learning settings, simulation tools, and more—that will make the idea of robots in everyday life a reality.
“Embodied AI will not only help address some of humanity’s biggest challenges but also create innovations which are currently beyond our reach or imagination,” says Geordie Rose, co-founder and CEO of Sanctuary AI. He further said that this kind of critical technology shouldn’t be developed in isolation, and that’s why they place a premium on partnering with stable companies like NVIDIA.
GR00T will additionally support new Nvidia hardware. The new computer Jetson Thor is keeping things moving in the Marvel Universe; its humanoid physical factor makes it ideal for performing simulation processes, generative AI models, and more. Although some experts still warn against using the word “general purpose” carelessly to describe these devices, Nvidia’s enthusiasm is encouraging and will likely speed up research in this category.
Nvidia notes, regarding the new silicon,:
To execute multimodal generative AI models like GR00T, the system on a chip incorporates a next-generation graphics processing unit (GPU) based on NVIDIA Blackwell architecture. The GPU, which uses a transformer engine, delivers 800 teraflops of 8-bit floating point AI capability. A high-performance CPU cluster, 100 GB of ethernet bandwidth, and an integrated functional safety processor make it easy to integrate and streamline designs.
Although general purpose is still a way off, making it easier for third-party developers to access will help close the gap significantly.
Two more important programs, Isaac Manipulator and Isaac Perceptor, were announced this week at the GTC robotics conference. For many years, manipulation has been an essential part of robotics. At the head of the pack were the enormous industrial robotic arms, which are standard equipment in the car industry. Even more agile and quick on their feet will be the generation that follows. It is to be expected that Nvidia would like to be involved.
“Isaac Manipulator offers state-of-the-art dexterity and modular AI capabilities for robotic arms, with a robust collection of foundation models and GPU accelerated libraries,” the company states. “It provides up to an 80x speedup in path planning, and zero-shot perception increases efficiency and throughput, enabling developers to automate a greater number of new robotic tasks.”
Notable companies that have already partnered with Nvidia include Franka Robotics, Yaskawa, READY Robotics, Solomon, PickNik Robotics, and Universal Robots.
Automatic mobile robots (AMRs) are also receiving some attention from Perceptor. This initiative continues Nvidia’s tradition of excellence in robotic vision processing. Particular emphasis is placed on “multi-camera, 3D surround-vision capabilities.” KION Group, ArcBest, and BYD are all registered participants.
There will be exciting competition for market share in the next years between mobile manipulators and humanoids, and Nvidia wants in on the action.
