The debate on local inference is shifting from raw speed to cognitive yield per unit of energy. A team at Stanford University's Hazy Research Lab, led by Christopher Ré and Azalia Mirhoseini, published Intelligence per Watt, the first systematic study quantifying the efficiency of local language models (LMs) on consumer hardware. The work introduces IPW, a metric that relates task accuracy to power consumption in watts, providing a unified indicator for evaluating the viability of on-device AI versus centralized data centers.

Add AlexTech.ai asPreferred Source on Google

88.7% of real queries handled without cloud

The research analyzes over 1 million single-turn and reasoning queries, comparing more than 20 local LMs (with fewer than 20 billion active parameters) across 8 hardware accelerators, including the Apple M4 Max and cloud GPUs. Results show that local models correctly answer 88.7% of real-world requests. The coverage of locally serviceable queries grew from 23.2% in 2023 to 71.3% in 2025, with an overall IPW improvement of 5.3x over two years. This trend confirms that local inference is no longer a quality compromise but a viable alternative for a significant portion of AI traffic.

The energy advantage of local accelerators

A crucial data point emerges from the architecture comparison: local accelerators achieve an IPW at least 1.4x higher than cloud GPUs running identical models. This gap highlights significant optimization headroom for consumer chips, which leverage data proximity and the efficiency of modern SoCs. The work fits into a broader context where energy efficiency is becoming the new bottleneck: as highlighted by a KAIST study on autonomous agents, energy consumption is pushing data centers beyond sustainability limits. Meanwhile, the open-source ecosystem accelerates with tools like Homebench for testing local performance and Perplexity experimenting with hybrid orchestration between PC and cloud.

Toward a new reference metric

The introduction of IPW signals a paradigm shift: performance is no longer measured solely in tokens per second, but in intelligence delivered per watt consumed. For IT decision-makers and developers, this means the choice between cloud and local is no longer a matter of brute capacity, but of economic and ecological efficiency. The framework proposed by Stanford becomes the reference for tracking this transition, with a public implementation supporting clients like Ollama and vLLM.