Preface: It’s October 2026, 4 months from now, GLM-6 was just released and it beats Fable-5.1 (the neutered re-release of the banned model) and it’s at par with Mythos on all the major benchmarks.
The US government can’t shut it down so they issue a series of actions to ban providers from serving it and companies from using it.
Z.ai and its affiliates may not directly or indirectly provide GLM-6 models, updates, inference services, managed deployments or technical support in the United States or to U.S. persons.
Amazon Bedrock, Google Vertex, and Microsoft Azure quickly issue statements saying they will comply and refuse to host the model for enterprise clients. The major aggregators like OpenRouter, Vercel, Cloudflare, TogetherAI and others agree to not list it. GitHub scrubs its platform of any traces of it.
Hugging Face is the last holdout, but eventually complies and removes all GLM-6 related models for download.
The scenario above is not the one we want, but a plausible outcome as the US and other governments try to react in a world where AI models are progressing at an exponential rate while policy continues to operate at a snail’s pace.
This type of outcome, or an alternative where the frontier of AI remains largely monopolized within a few centralized entities, is exactly why decentralized AI is important.
On the latest All-In pod: <a href=”https://x.com/DavidSacks?ref_src=twsrc%5Etfw”>@DavidSacks</a> <br><br>“I think centralization is the greatest threat of AI”<br><br>Couldn’t agree more, the question is what are we doing about it? <a href=”https://t.co/JVoP914KXK”>pic.twitter.com/JVoP914KXK</a>
This is a companion piece to my Proof of Useful Work primer – same approach, different corner of crypto-AI with some overlaps. I get into the hard problems to solve for decentralized AI to work, profiles of the projects I’m tracking in this space, a diligence framework, and where I land after going deep in this space.
Decentralized inference is inevitable
Following the scenario above, you may have already made the link to decentralized inference, but if not let’s continue to play this out.
Immediately after the GLM-6 model weights are released, copies spread throughout the internet – no amount of bans or remediation actions can mitigate the thousands of copies that now exist. Those copies will be served across decentralized inference networks where no central authority exists to issue actions against and no node can be banned that brings down the network.
I want to be clear on one point: I’m not here to argue whether this is a good or bad thing – if a new open weights model gets released that can cause serious damage via misuse, then I’m not suggesting we simply sit back and be ok with it. What I am suggesting is that it is inevitable that the model would be made available to those who don’t want to be censored from it.
That’s the core premise for decentralized inference – it’s a hedge against censorship of intelligence, whether that comes from governments or the frontier labs withholding it.
The additional value props like promises of cheaper tokens, verifiable inference and privacy are all secondary to this one core bet of censorship risk mitigation.
Now let’s get into it.
Decentralized inference is really hard
For most startups, solving one or two hard problems is a monumental challenge. For decentralized inference, the projects need to solve 4 really hard problems. How each project approaches these problems is what separates the vapor from substance, the alpha from the noise.
Hard problem #1 – running a model no single box can hold
The core idea is to create a swarm of GPUs and use pipeline parallelism to serve a model that people want. There’s a lot technical details here, but just imagine each node holds a slice of the weights and its own slice of the KV-cache – just enough that can fit on a 3090/4090 all the way up to H100s and above. Combine enough nodes together and now you can host a legit model like GLM.
Petals proved this method back in 2022 with BLOOM-176B across consumer GPUs in a BitTorrent-style swarm, but only at about 1 token per second. Obviously 1 t/s is not usable, so the innovation has focused on making models run faster.
The thing that kills it is the network. Inside a datacenter, GPUs talk over NVLink at terabytes a second. Across the public internet they talk at tens of milliseconds of round-trip. Decoding is sequential, so a naive swarm pays that round-trip on every single token.
The most common approach to this is speculative decoding. A small cheap draft model proposes K tokens, the big sharded model verifies all K in one pass through the pipeline, and you keep the longest run that matches. One expensive network crossing now buys several tokens instead of one.
Lots of technical nuance here, but the innovation is that we’ve seen this work up to about 30-40 tokens per second over real internet links. Good progress, but still unproven at both scale and the speed that users will really demand. I will reiterate – this is a hard engineering problem that takes real technical chops to solve.
Note: Serving inference is not just about FLOPs
Here’s the trap in comparing any swarm approach to cloud hosted models – people benchmark tokens per second and think that’s all that matters.
But production inference has to get a bunch of stuff right, none of which are about raw FLOPs:
- Time to first token versus inter-token latency
- Prefill versus decode – two phases with opposite hardware profiles
- KV-cache placement and transfer
- Streaming, continuous batching, and how utilization holds under mixed load
- Long-context behavior, cold-start and model warmup
- Node churn
The diligence point: when a project quotes a throughput number, ask what it is competing against. A centralized vLLM or SGLang deployment with disaggregated prefill and continuous batching is the real test, and it is a moving target getting faster every quarter. “We hit 30 tokens per second over the internet” is impressive engineering and still potentially uncompetitive.
Hard problem #2 – proving you got the model you paid for
If you don’t trust the node, how do you know it ran the model it claims and not a cheaper quantized stand-in? This is where providers can try to “game” the network, especially if a mining token is involved, to make it appear as though they’re serving the model you’re paying for, but in reality serve something cheaper.
There are 5 different approaches I’ve seen to tackling this hard problem:
- ZKML – a zero-knowledge proof of the forward pass. Cryptographic, airtight, and roughly 10,000 times overhead. About 150 seconds per token for Llama-3. Not happening at frontier scale anytime soon.
- opML – post the output with a bond, open a challenge window, and a fraud-proof bisects any dispute down to one step an arbiter re-runs. Near-native speed, but finality waits on the window, and it inherits the verifier’s dilemma: if checking costs more than the catch is worth, nobody checks.
- Deterministic re-execution – make inference byte-for-byte reproducible, then a dispute is just a byte-equality check. Under 2 percent overhead, secured by restaked ETH.
- Statistical fingerprints – hash or sample the computation cheaply and catch most cheating most of the time. Not absolute, but fast and GPU-tolerant, which is what a permissionless swarm needs.
- Live-weight proofs – sample the tensors actually resident in the serving process and check them against the approved model’s manifest. Verifies what’s loaded, not what’s emitted, at about 0.1 percent overhead. A genuinely different angle.
The trade-off is that you can realistically only get 2 of 3 things: 1) cryptographic integrity, 2) low latency, and 3) cost efficiency. ZKML gets you integrity at the expense of latency and cost. The rest get you latency and cost but settle for economic or statistical integrity.
The diligence point: ask what approach the project is using and why, understand what tradeoff they’re making and how that impacts the product.
Hard problem #3 – keeping your prompt secret
Proving the output is a different problem from hiding the input. In a sharded swarm, every node has to decrypt the activations to compute on them – encryption protects the wire, not the node.
Transformer activations turn out to be very invertible. CCS 2025 reconstructs input prompts from those intermediate activations at over 90 percent accuracy. ICML 2025’s “Hidden No More” gets near-perfect recovery and defeats the noise-and-permutation defenses a swarm often use.
The only robust fix is a different, heavier sequence-sharded scheme nobody in the consumer-GPU cohort has shipped (at least not that I’ve seen) so it remains a largely unsolved problem.
A swarm can advertise “no node holds the whole model” and still leak every prompt to any node in the path. “No node holds the model” was never a privacy property.
What actually delivers privacy is hardware or math, not topology. TEEs – Phala on GPUs, Darkbloom on Apple silicon, Venice’s Pro mode – move trust to a hardware root and attest it.
FHE computes on ciphertext and trusts nothing, but at a cost that isn’t usable for LLMs yet.
The diligence point: a project either has one of those or it doesn’t have privacy, no matter what the landing page says.
Note: Private does not mean trustless
The most important caveat in this whole section: a TEE does not remove trust, it relocates it. Your trust moves from the node operator to the hardware vendor, the firmware chain, the attestation service, and the enclave implementation.
So the real question is whose trust root you accept: chip vendors, a restaked validator set, a TEE network, or pure math.
Hard problem #4 – building a two-sided marketplace
The first 3 problem are all technical, this 4th problem is a business problem.
I’ve often wondered, for decentralized inference networks that serve open weights models, who is the ICP?
Most consumers are getting great value out of subscription plans – for $20-200 a month you can get a LOT of intelligence. Very possible those subsidized plans go away or limit usage in the future, but today if you’re trying to sell API-based pay as you go inference, the consumer is a hard sell.
Enterprises are not going to be big buyers either, maybe one day depending on how things play out with centralized models, but don’t count on enterprise buyers coming anytime soon.
So if consumers stick with their sub plans and the networks are not enterprise ready, that really leaves two categories: 1) startups and companies building inference into their product stack who by nature need API plans, and 2) autonomous agents seeking their own inference.
The startups category is a growing TAM and a niche that could be a wedge into a significant revenue business, but let’s recognize this puts a ceiling on value capture in the near term.
As for AI agents as buyers, that’s a bit more speculative – someone still has to pay for it in the near term. But the idea of agents performing autonomous work and procuring inference via crypto rails is a nice dream.
So that’s what makes this problem so hard – how do you aggregate meaningful supply of models people want while the target user base is unlikely to be big spenders on the network?
The one place this works is decentralized GPU providers. A host of projects including io.net, Akash, Render, Aethir, and Nosana have been at this for years and have created token-coordinated markets that rent out whole GPUs, or whole-model-per-node capacity, to whoever pays. So there is some precedent.
The diligence point: ask about the ICP of the projects and how they both acquire those target users while keeping the supply side happy. If it’s all based on speculative token upside, that’s your tell.
The flow of a decentralized inference request
If the problems above didn’t highlight the technical challenges, perhaps we should look at the flow from a user prompt from start to finish.
The decentralized inference provider has several extra steps beyond what a normal inference provider has to support.
Every hard question can be tied back to one of these steps:
- Where is the prompt visible in plaintext? (step 4 to 6)
- Where are the model weights visible, and to whom? (step 5)
- Where is the output bound to the claimed model? (step 8)
- Who can challenge a bad result, and what does it cost them? (step 9)
- What is the buyer’s actual remedy when the answer is wrong? (step 9)
- What is logged, and who can read it later? (step 10)
- Who signs the receipt, and is their signature worth anything? (step 8)
- And the master question: which of these ten steps is actually decentralized?
The diligence point: this is not a space where “crypto bros” can expect to make it as builders. There are no Aave forks to stand up. These are hard engineering problems, so you need to see real technical AI competence from the team.
Who’s solving these hard problems?
There are a LOT of projects that fall into the “decentralized inference” category in some way or another. Most are not solving all 4 problems equally and tend to lean towards focusing on one core problem with elements of the others baked in.
Project Profiles
Petals was the pioneer of decentralized inference. Some big brain researchers showed in 2022 that BLOOM-176B could run across consumer GPUs in a BitTorrent-style network, with each participant hosting a few transformer blocks and passing activations down the chain. It proved that pipeline parallelism over weak internet links could be survivable.
Its importance is conceptual, not commercial. Petals did not solve incentives, privacy, or monetization. It had the architecture that crypto later wrapped tokens around, but not the economic layer.
This is why it’s important to know this project – when you see something that is essentially Petals + a token, it’s most likely a larp.
Dolphin is the team behind the Dolphin family of uncensored open models that have over 5m downloads on Hugging Face. That origin matters – Dolphin did not start as “we have a token, now we hope to find a use case.” It started with models people already used and then wrapped a network around that demand.
Dolphin is more than just a decentralized inference network (it deserves a more in-depth piece) but the inference product is a peer-to-pool network where GPU owners, including gamer and prosumer GPU owners, contribute idle capacity into model-specific pools. Requests go into a pool and are assigned to nodes that can serve the model. That design is made for elastic, unreliable supply – the person can take back their 4090 when they want to play a game – and for uncensored / specialized open-model demand that frontier labs are structurally unlikely to host.
The differentiated technical piece is live-weight proofs. Instead of hashing a model file at load time and hoping it stays there, Dolphin samples the tensors actually resident in the serving runtime and checks them against the approved model manifest. That verifies what is loaded, not what is emitted, at about 0.1 percent reported overhead – extremely efficient. It layers that with logprob fingerprinting, software-integrity checks, and account-wide bonding where an operator has to build a slashable bond before liquid claims unlock.
The capacity signals are real – 3.2 billion plus generated tokens and about 9,400 tokens per second of sustained bandwidth by the project’s own figures. So Dolphin is a product-first, founder-credible execution bet.
Inference.net, formerly Kuzco, is a decentralized inference network that is one of the more mature attempts to verify model execution in the wild.
The distinctive mechanism is LOGIC, a logprob-based verification approach using statistical tests to catch model substitution at decode time. In plain English: the network checks whether the probabilities behind the emitted tokens look like they came from the claimed model, rather than a cheaper substitute. This sits in the same broad family as Proof of Logits and other statistical fingerprints. It is not a hard cryptographic proof, but it is cheap and practical on heterogeneous GPUs.
Inference.net has been in production for roughly 18 months and has a reported fleet in the 5,000 to 10,000 GPU range depending on source, although the figures are largely self-reported from what I’ve found. It is one of the few projects with both a verification primitive and meaningful operating history.
Morpheus is a decentralized routing-and-rewards layer that connects users to open-source LLMs hosted by third-party providers. The user-facing pitch is an OpenAI-compatible API plus a “smart agent” wrapper that can connect an LLM to a Web3 wallet and execute on-chain actions. Underneath, it uses the Lumerin proxy-router for session bidding, payment in MOR, and prompt / response forwarding.
The strongest technical piece is TEE-backed provider verification: Phase 1 is live with Intel TDX plus NVIDIA GPU attestation, no prompt or response logging at the protocol level, and a roadmap toward a fuller trust chain.
The traction is what I’d keep eyes on here, especially as it relates to MOR emissions and real evidence of external inference demand.
Chutes is Rayon Labs’ serverless inference platform on Bittensor Subnet 64. To the user, it looks like an OpenAI-compatible API for running models. To the builder, the unit is a “chute” – a Docker-packaged workload that gets deployed onto a backend of Bittensor GPU miners.
On the positive side, it has distribution via OpenRouter, an easy developer surface, and large headline scale. There are some legit real models being served at competitive pricing. However, it seems to have some gaps on verification and privacy that remain to be solved. GraVal proves the GPU is real, not that it ran the claimed model, and the per-token model-binding proof is single-source and unaudited. Privacy is opt-in TEE only, so default mode still exposes the prompt to the operator.
c0mpute is the flashy new project: a Solana-native, open-source decentralized inference project whose Shard engine splits frontier-scale models across consumer and prosumer GPUs over the public internet.
The technical artifact is real and independently checkable. c0mpute published a GLM-5.2 744B NVFP4 demo across 7 RTX PRO 6000 GPUs in 6 US states, with WAN round-trips in the 22-75 millisecond range and about 30 tokens per second. It also showed gpt-oss-120B across 3 RTX 4090s at about 40 tokens per second.
There is some substance here: GPU UUIDs, public IPs, regions, per-edge latency, output token IDs, hashes, engine hashes, and determinism checks have all been made public.
But, we need to recognize this is still very early days. The repository was only days old at the time of writing, the founder is pseudonymous, the $ZERO token is a pump.fun microcap with fragile liquidity, and value accrual is speculation based at this time.
Here’s what I’ll be watching for: real third party validation on the 744B run under concurrent load, and with future tests. Also, the privacy pitch is not defined because each shard sees activations. So c0mpute is impressive execution and white-space positioning, but needs more time to prove that it’s solved the technical challenges and can scale.
Parallax is Gradient Network’s P2P distributed LLM inference framework, part of its “Open Intelligence Stack.” Think of it less as an app and more as an engine: pipeline-parallel sharding across heterogeneous consumer GPUs and Apple Silicon so individuals, organizations, or small nations can run “sovereign clusters” without hyperscalers.
Relative to c0mpute it has the same broad architecture, same WAN pipeline-parallel serving thesis, same Solana orientation, but Parallax has a bit more institutional credibility with a $10m seed led by Pantera and Multicoin, and a visible team.
The honest question is whether Parallax becomes a substrate others build on, like llama.cpp or vLLM for decentralized serving, or whether the commercial value moves to the apps and routers above it. It also lacks a real privacy fix as far as I can tell.
Darkbloom is a project born out of Eigen Labs’ that lets users turn idle Mac compute into a private inference marketplace – i.e., you rent out your Mac to serve models to other users.
A good analogy is Apple Private Cloud Compute, but rebuilt over untrusted consumer hardware. A coordinator routes requests to operator-owned Macs, the Mac runs the whole model locally through MLX, and the privacy guarantee comes from Secure Enclave attestation, hardened in-process execution, and end-to-end encryption.
The “whole model per Mac” detail is the key. Darkbloom is not a sharded swarm – it does not pass activations across a chain of anonymous nodes, The attestation stack is quite serious and no operator can read prompts or outputs unless the Apple-rooted trust model itself fails.
The traction is early but real enough to be interesting. It moved from research preview to public alpha on May 26, has about 250 peak live nodes, thousands of signups, 600 million plus tokens served, Stripe Connect live, Solana USDC funding, and an OpenRouter listing that is currently free and subsidized.
Live models are still modest but this is one of the cleanest signal in the field that decentralized does not have to mean tokenized.
Hyperspace Pods is the private-cluster version of the consumer P2P mesh idea. It lets families, friends, startups, or small organizations form private AI clusters from their own laptops and desktops, auto-shard open models across devices, and expose a single OpenAI-compatible endpoint.
This is still emerging so I would just keep it on the radar and watch for tokens-per-second benchmarks on real multi-device Pods, active cluster counts, and whether “rent out idle capacity” becomes a real market or remains a local-cluster feature.
MeshLLM is a permissionless P2P inference mesh introduced by Jack Dorsey and built by Block-affiliated contributors. Nodes discover each other over Nostr, gossip through a mesh with no central servers, and expose OpenAI-compatible APIs. It is built on llama.cpp and is Apache-2.0 licensed (copyright Block, Inc.).
The thing that makes MeshLLM worth mentioning is the institutional roots. Block has consumer-scale distribution, Dorsey has spent years pushing Nostr as a decentralized coordination layer, and MeshLLM extends that thesis into AI compute. It is closer to BitTorrent than Bittensor: protocol-first, tokenless, permissionless, and censorship-resistant.
Venice and the Inference Resellers
Any discussion of crypto and inference has to give a shoutout to Venice – it’s the poster child for the entire sector in terms of finding pmf and building a viable business with crypto incentives. They effectively solved problems 3 and 4, at least to some degree.
Venice is not decentralized inference in the swarm sense – it’s a centralized, privacy-laddered consumer proxy with TEE-plus-end-to-end-encryption Pro mode.
But it has the customers, it’s uncensored, and it has become the foundation of a sub-sector of projects reselling surplus inference. Those resellers decentralize the demand aggregation and settlement, but none of the compute.
Venice sells staked-DIEM holders a perpetual one-dollar-a-day API credit. A lot of that credit expires unused. So a reseller lets credit holders sell their surplus allowance for USDC, routes buyer demand to the cheapest seller, and takes the spread. The “discount” is the gap between cheap tokenized Venice credit and retail API pricing.
Several of these projects now exist: UsePod, AntSeed, Surplus Intelligence, CheapTokens, and Built in Venice. Some of these have much bigger ambitions bolted on and are using the surplus as a wedge to build scale.
The reason I’m calling these out here is that they often get categorized as decentralized inference, and they are in some ways, but not in the purest form which involves aggregating decentralized compute and serving models across a disparate network of providers.
UsePod is an early inference marketplace that routes requests to the cheapest eligible provider and settles in USDC, SOL, or Stripe. The origin story is the key: founder Chris Gilbert started by reselling Venice DIEM staker inference credits, then generalized the wedge into a broader inference router. That makes UsePod part of the Venice-resale cohort, but it is also trying to turn the wedge into something bigger.
AntSeed is the Base-native, P2P version of the same Venice-adjacent inference-resale pattern. Providers connect upstream APIs or local models, announce capacity over a BitTorrent-style DHT, and buyers route requests through encrypted WebRTC connections. It exposes local OpenAI- and Anthropic-compatible APIs, so developer tools can point to it.
What makes AntSeed different is the protocol / P2P transport thesis. It sells a decentralized OpenRouter story: no central listing approval, no company server in the request path, no platform custody of provider earnings, and USDC settlement direct to provider wallets.
Surplus Intelligence is the order-book version of the resell model. Buyers hit an OpenAI-compatible endpoint for discounted models, sellers plug in upstream prepaid API credits, and Surplus routes demand to the cheapest seller and settles in USDC. It owns no GPUs, hosts no models, and runs no inference. The “surplus” is financial – unused prepaid credits – not physical idle hardware.
Where decentralized inference wins, and where it loses
The cost thesis only works if you separate latency from throughput. They are different products, and decentralization is a tax on one and a feature on the other.
Where decentralization is a tax (centralized wins):
- ChatGPT-style interactive chat
- Real-time coding agents
- Low-latency voice
- High-frequency tool-calling loops
- Any enterprise app with a strict p95 latency SLA
- Frontier-scale dense models at competitive latency – nobody decentralized serves a 200-billion-plus dense frontier model competitively today
Where decentralization can be supply aggregation (it can win):
- Synthetic data generation
- Offline evals and benchmarking
- Bulk embeddings
- Batch RAG and document processing
- Long-running agent research jobs and backtests
- Image and video generation queues
- Non-urgent open-model inference where idle-hardware marginal cost is near zero
The easiest framework: where latency matters, decentralization is a tax. Where throughput matters, decentralization can be supply aggregation.
Hidden value: the data loop
In one of my previous articles I wrote about the moats in AI agents and a core thesis that it’s not the harness itself that is the moat, but the data the harness collects.
It is worth noting that decentralized inference networks can also collect valuable data such as synthetic training data, preference data, agent traces, eval outputs, fine-tuning data, RL environments, tool-use trajectories.
That data can feed decentralized training systems – Nous Psyche, Prime Intellect, Gensyn-style networks – which produce updated open-weight models, which flow back into the inference networks.
The long-run stack is not “decentralized training” or “decentralized inference” as separate bets. It is a loop: inference generates traces, traces become training data, training updates open models, and updated models flow back into inference.
The best decentralized inference networks will incorporate this as strategy, and I expect to see more convergence between decentralized training and inference projects.
Diligence checklist: seven questions is all you need
Forget all the technical jargon – you can judge any project in this space on how they answer seven simple questions.
- Is it actually decentralized, and if so, what layers? Many will slap “decentralized” on their project just because it has a token while the product itself is centralized.
- Can you trust the output came from the model you paid for? This is where correctness lives – determinism, proofs, fingerprints, or nothing.
- Is it actually cheaper than centralized, after the token and the coordination overhead are paid for? Not cheaper in theory. Cheaper in production.
- Is your prompt actually hidden from the operator? TEE or FHE, yes. Sharding, no.
- Does it hold together when the nodes are flaky and scattered across the internet? Orchestrating unreliable, heterogeneous hardware at scale is its own discipline.
- Is anyone actually paying for it, in a form they couldn’t get cheaper centrally?
- Does the team have real AI technical competence? Probably the most important question. If it’s a team of crypto trenchers, then just know it’s likely more hype than real.
One piece of advice: watch out for elegant technical solutions that don’t have any credible plan for distribution.
Where I land
I’m quite bearish in general on categories that are only attractive/relevant to crypto natives (a largely unattractive TAM imo). I want to see projects that are attractive to non-crypto natives and keep the crypto mechanics mostly under the hood.
Decentralized inference as a category is one of the few sectors in crypto that has real breakout potential – everyone wants inference and it can be served up just like traditional providers or even route through existing platforms like OpenRouter for a completely seamless experience. What is important is cost, performance and privacy
My advice – back the projects that are precise about which layer they decentralize and understand who are the buyers of their decentralized inference. Avoid the projects just selling “decentralized AI” as a slogan with a coin attached.
Disclosure: I hold tokens of some of the projects mentioned in this article. I was not influenced by, or received any compensation from, any project mentioned in this article – the framing and judgments are my own.
If you’re building at the intersection of AI and crypto with a real product underneath, my DMs are open.