Nvidia recently announced an astonishingly cool collection of universal AI models, libraries, and infrastructure to support and attract robotics developers. This announcement marked another major milestone in the company’s long commitment to developing cutting-edge artificial intelligence technologies. Sanja Fidler, vice president of AI research at Nvidia, emphasized the importance of this initiative in the current technological landscape.
The unveiling took place during the Consumer Electronics Show (CES) in January, where Nvidia introduced its Cosmos family of AI models. These developments showcase the company’s evolution from a modest research lab to a powerhouse in the AI sector, now boasting a workforce of over 400 professionals. Bill Dally, Nvidia’s chief scientist, was instrumental in this pivot.
The Journey of Nvidia’s Research Lab
Tarjan Professor Bill Dally began consulting for Nvidia in 2003, while still at Stanford University. First, his main project was on ray tracing, a rendering technique that’s central to the field of computer graphics. In 2009, Dally became head of the research lab, which had just a dozen people working there at the time. Under his leadership, the lab grew by more than half again while diversifying its research areas and policy impact.
“At Nvidia, we learn to be really good at guessing what’s going to make the most positive impact for the company,” said Dally. “We’re constantly seeing exciting new areas, but some of them do great work. We have trouble saying if we’ll be wildly successful at this.”
The expansion into their new, much larger venue has obviously come from Dally’s vision and commitment to making innovation happen. The code and systems they are building could very well revolutionize the world and how we approach computing, he argues. “We said this is amazing; this is gonna completely change the world,” he remarked when discussing the potential applications of their research.
Advancements in AI Models and Robotics
The Cosmos family of AI models was just released a few days ago. This acquisition serves as a watershed moment for Nvidia, as it wraps up its stronghold in the AI space and autonomous transportation. Sanja Fidler joined Nvidia in 2018. Prior to that, she worked with a group of students at MIT to create simulation models for robots. She stressed the cultural fit in spades at Nvidia. It was just such a great fit for the subject matter, and you know, it really matched the company culture so perfectly,” she explained. Jensen [Huang] said to me, come work with me, not with us, not for us, you feel me?
The Cosmos models will augment robotic capabilities by using state-of-the-art AI algorithms. By using videos captured from robots and self-driving cars, Nvidia developed 3D models and simulations through its Neuric Neural Reconstruction Engine. This groundbreaking technology, initially revealed back in 2022, powers the transformation of 2D imagery into extensive 3D meshes.
Fidler elaborated on the direction of their research: “You go from rendering means from 3D to image or video, right? And we want it to go the other way.” This strategy not only reinforces their upmost priority of expanding the limits on technology in robotics and AI.
The Future of Nvidia and AI Research
As Nvidia deepens its commitment to R&D, Bill Dally is looking confidently toward the future. “We’re making huge progress, and I think AI has really been the enabler here,” he commented. The advancements made by Nvidia’s research lab not only contribute to the company’s growth but have broader implications for various industries that rely on AI technologies.
Dally’s leadership has been crucial in fostering a culture where creative ideas can thrive and experiment. He stated, “I think everybody’s always searching for the place in life where they can make the biggest contribution to the world. And I think for me, it’s definitely Nvidia.” With an eye towards the future and a penchant for research and development, Jensen’s vision has turned Nvidia into one of the most powerful tech companies in the world.
