Large models also have full-blooded and shrunken versions: Switching cloud vendors could halve performance

According to Perceptual Beating monitoring, model evaluation agency Artificial Analysis has launched an API Accuracy Ranking, testing how much capability remains when the same open-source model is transferred to different service providers. The first batch tested GLM-5.2, gpt-oss-120b, and DeepSeek V4 Pro, covering a total of 44 API endpoints.

The service providers include AWS, Microsoft Azure, Google Vertex, Cloudflare, as well as Fireworks, DeepInfra, CoreWeave, SiliconFlow, and other AI inference platforms. Artificial Analysis deployed its official model, assigning a score of 100%, and then tested each API using the same criteria.

The results showed that the worst-performing endpoint of GLM-5.2 achieved only 52%, nearly halved. The scores for gpt-oss-120b ranged from 70% to 101%. DeepSeek V4 Pro was the most consistent, with all 9 service providers scoring between 97% and 107%, with the official API scoring the highest.

The differences mainly stem from quantization, output length, inference configuration, and tool invocation handling. Under the same model name, the actual capabilities may vary significantly, so selecting an API should not be based solely on price and speed.