Imagine a world in which Frontier AI Labs operated like energy distributors and most users had the equivalent of solar panels contributing inference to the grid. It could completely change the economics of knowledge production. It could be a world in which an AI subscription isn’t a product you buy; it’s a net-settlement agreement between your hardware and a Frontier lab’s cloud. To achieve this, Frontier labs need to create tied local models that offload some inference tasks to the users hardware. This notion would fit neatly with the increasing OpenClaw related habit of creating local memory for users skills, tasks and data. Finally, this local inference capacity could conceivably be used for other users’ tasks by the Frontier lab, where one user gets paid in reduced inference bills, the inference equivalent of the solar panel: an inference grid.
The analogy here is rooftop solar. When households began installing solar panels, the utility companies saw a threat. Distributed generation does reduce grid load and compress margins on the commodity electricity business. Some utilities responded by lobbying against feed-in tariffs and fighting net metering schemes.
But the economics are strong. The households installing panels had a sunk asset, hardware they own. They could use it to cut their bills and sometimes get paid for contributing to the electricity grid. Successful grid operators stopped treating rooftop generation as competition and started treating it as distributed infrastructure they could orchestrate. They could lower their own generation costs using users’ production.
The CPU/GPU on the laptop in your bag is like a solar panel. It was bought for other purposes. It sits mostly idle between tasks. Its marginal cost of inference is close to zero. The Frontier lab that figures out how to treat it as distributed infrastructure - rather than as a threat to per-token billing will change the economics of the industry.
A closed-local model is the first move. Many people already use small open models for limited inference tasks. Why not have a Frontier lab provide a closed local model (ie tied to their larger infrastructure) to run on client hardware, handle routine tasks, second brain processing locally, and escalate to cloud inference when the task demands it. The client relationship is retained. Switching costs rise substantially - that moat the Frontier Labs are so desperate to build. And the lab stops bearing the compute cost of everything a user does, including the tasks that never needed a data centre in the first place.
A MacBook Pro today can run a capable 8-14 billion parameter model at 60 to 80 tokens per second. It is a miracle of engineering currently being used by many knowledge workers as an overpriced typewriter. For summarisation, classification, document processing, code review, and routine correspondence, the quality is comparable to Frontier model’s performance for a lot of what a knowledge worker actually does in a day. The tougher stuff can go to the big model. The marginal cost is electricity. But currently, no Frontier Lab operates such an own brand local-central hybrid model.
As well as distributed inference, this local model could work well with the increasing volume of AI generated files used in “second brain” memory structures. Partly off the back of the OpenClaw explosion, users are building second brains - local folders containing meeting notes, project history, correspondence drafts, half-formed ideas, personal context accumulated over years. This material is, by definition, personal. Nobody wants it on someone else’s server. It cannot be meaningfully sanitised without destroying its value. The local model is its natural processor: it handles the material natively, escalates only what it must, and the personal archive never leaves the machine.
These closed hybrids don’t yet exist. Instead, Frontier Labs are currently trying to win the compute race with massive spending on training and inference. This may be a winner’s curse. The compute race is not irrational. Frontier model quality scales with training and inference compute, and training and inference compute requires infrastructure only a handful of organisations can afford. The labs that fall behind on capability fall behind permanently. However, the winner may find themselves stuck with a very expensive , fast depreciating asset.
The current compute race has the frantic energy of a digital gold rush, except the miners are buying the shovels for $100,000 a piece and the gold is a depreciating asset with the shelf-life of an open avocado.
For the economy as a whole, the data centre race is dysfunctional. The capital consumed has an opportunity cost. Every billion committed to a data centre is a billion not going to drug discovery, climate infrastructure, or agricultural technology. It is the research that does not happen, the tools that are not built, the problems that wait another decade for serious attention.
Back to solar: in many energy markets, households with solar panels sell surplus power into the grid. The grid operator buys it, redistributes it, and earns from orchestrating the network. The household’s sunk asset becomes an electricity generation node in its infrastructure. Having the panel not only produces electricity for yourself, but can also do it for others too.
The inference grid could work in the same way. When your machine has idle capacity - most of the time, for most users - the lab routes other workloads to it. Cryptographically isolated, sandboxed, invisible to you. You see resource consumption. You do not see content. The client whose inference runs on your hardware sees only their result. The lab sees the outcome and earns the margin. The user meanwhile, has their inference bill reduced by the Frontier Lab.
There are significant potential first-mover advantages. The inference grid will have scale and network economies. A distributed network of credentialed nodes takes time to build and will be difficult to replicate quickly once an incumbent has it. There are issues of latency and security that will need resolving.
Additionally, there is an assumption that because Anthropic, OpenAI and Gemini are ahead in the technical stakes right now and have access to the best GPUs, this will continue. It should be noted, however, that the Chinese are world leaders in making solar panels, and increasingly small local LLMs.