NVIDIA Aims to Build World's Most Powerful Open-Source Model, Nematron 4 with at Least 1 Trillion Parameters

According to Watchful AI monitoring, NVIDIA is intensifying its efforts on Nemotron 4, aiming to make it one of the world's most powerful open models. The Information cited multiple project employees stating that the flagship model is expected to have at least 1 trillion parameters, roughly twice the size of Nemotron 3 Ultra. Specific specifications and release date are yet to be determined, and the final training has not yet begun.

NVIDIA hopes to drive GPU demand with a more powerful open model. The company has significantly increased its in-house model's computational power investment in recent years. As of April this year, future cloud service procurement commitments have increased to $28 billion for the next few years, about three times the amount from a year ago, although this also includes other R&D purposes.

Nemotron 4 has also brought on board a group of external companies for joint development. Cursor, Mistral, Thinking Machines, Reflection AI, among others, will provide training data, evaluation, and model design, while Prime Intellect has contributed 300,000 simulated environments.

The issue is that many of these companies are already NVIDIA's customers or invested enterprises. Top model companies like OpenAI are also among its largest GPU buyers. Now, NVIDIA itself is pushing into cutting-edge open models, seeking to both grow the open model market and compete with its own customers and partners.