Despite Beijing’s aggressive push for technological self-sufficiency and recent mandates requiring state-funded data centers to use local hardware, China’s most advanced artificial intelligence models are still primarily being trained on Nvidia chips. According to sources at major Chinese large language model (LLM) developers, the prohibitively high cost of switching to local semiconductors continues to hamper the broader adoption of domestic alternatives in the private sector.
While domestic hardware most notably Huawei Technologies’ Ascend series, continues to advance in raw computational power, changing the underlying chip architecture presents a steep and expensive engineering bottleneck for AI developers. “Training LLMs on Nvidia chips for now remains the norm among Chinese AI developers,” a person familiar with the industry told the South China Morning Post.
This enduring reliance highlights the profound difference between achieving hardware parity and breaking a deeply entrenched software monopoly. While Chinese chipmakers are making rapid strides in silicon design and manufacturing, they are struggling to replicate the frictionless developer experience that has made Nvidia the undisputed king of the global AI boom.
The CUDA Lock-In Effect
The core hurdle delaying the switch to domestic chips lies not in the silicon itself, but in the software ecosystem that surrounds it. Nvidia’s Compute Unified Device Architecture (CUDA) platform has been the industry standard for AI development for nearly two decades. The vast majority of the world’s open-source AI libraries, frameworks, and developer tools are optimized specifically to run on CUDA.
By contrast, Huawei’s alternative software stack the Compute Architecture for Neural Networks (CANN) requires developers to rewrite and optimize large amounts of existing code to achieve comparable performance. This translation process is neither simple nor cheap.
“Our existing training pipelines are reliant on CUDA,” said James Wang, a developer working on AI models at a research institute affiliated with a Shanghai-based university. “CUDA code cannot run directly on Ascend and requires extensive rewriting.” Wang estimated that migrating his team’s existing workflows from Nvidia hardware to Huawei’s Ascend chips would add at least 50 percent in both time and engineering costs. For fast-moving AI start-ups racing to release the next generation of models, that delay is often unacceptable.
Training vs. Inference
The reluctance to abandon Nvidia is particularly acute during the model training phase. Training a large language model from scratch requires orchestrating thousands of GPUs to work in perfect synchronization for weeks or months at a time. Any hardware instability or software inefficiency during this process can ruin a multi-million-dollar training run. Nvidia’s mature ecosystem provides a level of reliability and fault tolerance that domestic alternatives have yet to fully match at scale.
However, the dynamic shifts significantly when models move from training to inference the process of running a completed model to generate responses for users. Inference requires far less synchronized computing power and is less dependent on the deep optimizations provided by CUDA.
As a result, many Chinese AI companies are adopting a hybrid approach: utilizing stockpiled or smuggled Nvidia chips (such as the H800 or H20) for the critical training phase, while increasingly deploying domestic chips like the Huawei Ascend 910B for inference tasks. This strategy allows them to manage costs and comply with government pressure for localization without risking the core development of their flagship models.
The Cost of Decoupling
The persistence of Nvidia’s dominance in the private sector stands in stark contrast to the government’s recent directive requiring all state-funded data centers to use exclusively domestic chips. This regulatory divergence is creating a two-tiered AI infrastructure in China.
State-backed research institutions and universities are being forced to absorb the high transition costs and engineering delays associated with migrating to platforms like CANN. Meanwhile, private tech giants like Tencent, Alibaba, and ByteDance, while publicly supporting domestic chipmakers, continue to quietly leverage their massive financial resources to secure access to Nvidia hardware, often through offshore data centers or secondary markets, to maintain their competitive edge.
Breaking the CUDA lock-in will require more than just government mandates. It requires building a comprehensive, developer-friendly software ecosystem that rivals Nvidia’s decades of investment. Until domestic players like Huawei can offer a seamless software experience that doesn’t impose massive rewriting costs on developers, Nvidia’s chips will remain the foundational engine of China’s most advanced AI ambitions.
The timeline for this transition is not measured in months but in years. Nvidia has spent over two decades cultivating the CUDA ecosystem, investing billions in developer education, open-source libraries, and enterprise support infrastructure. Huawei’s CANN platform, while improving rapidly, is still in its relative infancy by comparison.
The government’s domestic chip mandate may succeed in redirecting state-funded procurement, but the private sector’s continued reliance on Nvidia reflects a rational engineering calculation: when the goal is to build the world’s most capable AI model, every percentage point of training efficiency matters. Until that calculation changes, the gap between China’s official self-sufficiency narrative and the reality of its AI labs will remain wide.