You Don't Need to Run Everything Locally | Agentic & Distributed Systems, DeFi, and Compute Economics

Local AI is not a hardware contest. Run locally what fits, borrow the datacenter for what does not, and keep the agent either way.

I went back through the hardware behind a normal AI answer.

The gap is ridiculous. On one end, a laptop can summarize a private document without sending it anywhere. On the other, one datacenter system can pull more than 10 kilowatts before cooling and facility overhead.

Both get called “using AI.” That hides the part people actually need to understand.

The bill

I learned the cloud side of this the expensive way.

When I started running OpenClaw seriously, my Anthropic bill went from $25 a day to $800 a day. Some of that bought useful work. Some bought loops, retries, apologies and code that never worked.

A $10 or $20 chat subscription does not prepare you for that. A chat product is priced for a person asking questions. A persistent agent can work all day, call tools, retry failures and use APIs while nobody is watching.

That is a different economic object.

The hardware behind the biggest models is also a different object. An NVIDIA DGX H200 has eight H200 GPUs, 1,128GB of GPU memory and a maximum system draw around 10.2kW. At $0.30 per kilowatt-hour, running at that ceiling for 30 days would put power alone around $2,203. Cooling, networking, storage, redundancy and the machine itself come after that.

That is a scenario, not everybody’s bill. Most people should not buy a datacenter.

The point is that one clean answer can hide an industrial supply chain.

The wrong question

People ask me which model they should run locally.

I think the better question is which work should stay local.

Writing from notes. Summarizing documents. Classifying files. Extracting fields. Searching a private archive. Drafting a reply that a person will review.

Those jobs do not automatically need the biggest model available. They need a model that clears the quality bar, access to the right context and a system that knows what it is allowed to do.

The model is only one part of the job.

Local AI is not a purity test. It is a spectrum of control, privacy, portability and ownership.

A model running on your laptop gives you more control over where inference happens. It does not make the whole workflow secure. The files may still sync to a cloud drive. Credentials may sit in plaintext. Logs and backups may leak the same data you kept away from the model provider. A connected tool can still send, delete or spend.

Local inference shrinks one boundary. It does not erase the others.

The machine you have

The first tier is boring on purpose.

Use the machine you already own.

Smaller models can handle writing, summaries, document questions and private everyday help. They can classify incoming material, prepare a draft and keep routine context close to you.

The quality will not match the best cloud model on every job. That is fine.

The test is not whether the local answer wins a leaderboard. The test is whether it clears the bar for this job without creating more review work than it saves.

If it does, use it.

If it does not, send the hard part somewhere stronger.

Hardware should never become the admission price for using AI well.

The practical middle

The middle is getting more interesting.

As of September 2026, Gemma 4 models and Qwen 3.6 checkpoints such as the 27B and 35B-A3B are the kind of open-weight options I would examine for a serious local setup. They are big enough to do useful work and small enough to belong in a workstation conversation.

That does not mean every version fits every machine.

Weights need memory. Quantization reduces that memory, but it can change quality and speed. Context uses more memory. The operating system and everything else on the box need room too. Concurrency changes the answer again.

“Can load” and “can run well all day” are not the same claim.

This is why I do not like universal hardware recipes. A configuration that works for one quantization, context length and workload may fall apart when the job changes.

Buy for the work you can name. Test before you scale.

The sovereign workstation

The next tier is not a better laptop. It is a workstation built around memory.

GLM-5.3-Flash is a good example of where this gets serious. Its official model card lists 320 billion total parameters with 18 billion active. The weights are released under MIT, and the project publishes local serving paths for vLLM, SGLang, Transformers and other runtimes.

DeepSeek-V4-Flash-0731 is another. The released checkpoint is listed around 304 billion parameters. Its preview architecture was reported at 284 billion total with 13 billion active and a one-million-token context window.

These are not casual laptop models. With suitable quantization and enough memory, they are targets for a high-memory sovereign workstation.

Target is the important word.

I would not promise that one precise $10,000 configuration runs every version at the speed, context and concurrency somebody expects. Model files, quantization, runtime support and memory overhead decide that. The honest path is to choose the workload, choose the exact checkpoint, calculate the memory budget and test the stack.

Ownership does not remove engineering.

The datacenter

Then there are models like Kimi K3.

Its official card lists 2.8 trillion total parameters, 104 billion active, 896 routed experts and a one-million-token context window. The weights are available under a custom Kimi K3 license.

That is open weight. It is not consumer hardware.

Open does not mean small. Downloadable does not mean practical on the box under your desk. Active parameters matter for compute, but the whole model still has to live somewhere. Quantization helps. It does not turn 2.8 trillion parameters into a laptop model.

This is where the cloud still earns the call.

If a job needs that capability, borrow the datacenter. Pay for the peak. Do not rebuild your whole operation around owning a machine that makes no economic sense for your actual volume.

I am not anti-cloud. I am anti-dependence without an exit.

The agent

The mistake is treating the model as the thing you are building.

The model should be replaceable.

Your agent identity, memory, context, rules, tools, approvals, records and workflows should survive when the model changes. That is what I care about with cypher.claw at cYpher.camp.

A writing job might use a small model on the machine you already own. A harder coding job might route to GLM-5.3-Flash on a workstation. A long, high-value research job might borrow frontier cloud capacity. The agent should still know the job, the rules, the history and where a person must approve the consequence.

Models are not interchangeable. Prompts change. Tool behavior changes. Quality changes. Moving is work.

But moving should not mean rebuilding the operator from zero.

That is the difference between owning a workflow and renting a chat window.

The rule

I do not want people buying hardware before they know what it is supposed to do.

Start with the recurring job. Decide what data can leave. Set the quality bar. Measure the cost of a bad answer. Then use the smallest system that clears it.

Keep private, repeatable work local when it makes sense.

Use the cloud when quality, speed or scale earns the call.

Keep consequential actions behind a person, no matter where inference runs.

Run locally what fits. Borrow the datacenter for what does not. Keep the agent either way.