OpenRouter upgrades inference routing: pay 6% more for about 17% faster response times

Beating AI News Flash: OpenRouter announces an upgrade to its default inference routing algorithm. Previously, inference providers were primarily selected based on price; now it comprehensively considers speed, stability, and actual invocation costs, no longer blindly choosing the cheapest provider.

Officially, over 24 million requests were tested. Results show that time to first token was reduced by 16.9%, total request latency was reduced by 19%, and the proportion of requests encountering rate-limiting errors dropped from 7.1% to 3.8%. The trade-off is an approximately 6% increase in cost per thousand requests.

The new algorithm is now live, and users do not need to adjust their configurations. This optimization targets the selection of inference providers behind the same model, rather than the model's own capabilities.