DeAI I: The Tower & the Square

AI x Crypto (Part 1)

Part 2 – Seizing the Means of Production (Infrastructure)

Part 3 – Composable Compute (Middleware)

Part 4 – The Agentic Economy (Apps)

The Tower & the Square

In writing this piece, I kept coming back to themes from Niall Ferguson’s “The Square and the Tower” which traces the power dynamics between traditional hierarchies (the Tower) and networks (the Square) throughout history. While hierarchies have traditionally been the dominant forms of organization, networks have played a crucial role in shaping societies, and their rise – the Enlightenment, the American & French Revolutions, the internet, etc – have often marked profound shifts in societal organization.

The 21st century is primed for a continued struggle: the state, the corporation, the elites vs. the network, the interwebs, the proles. The hierarchy and the hivemind. The Tower and the Square.

Much of post-cold war history has been a story of the rise of networks: globalization, markets, liberalism, the internet, social media, crypto. Perhaps most symbolically, in 2021 we witnessed a social network suspend the sitting President of the United States.

However, recent years have seen the Tower reassert itself. Geopolitics are once again resurgent over economic efficiency. The tools which promised open communication have been trained inward, honing Orwellian-level surveillance of our every click. The “open internet” has fragmented into four increasingly extractive and censorious “walled gardens”.

History chronicle’s countless pendulum swings between centralization and decentralization, networks and hierarchies, anarchy and autocrats, but this time feels different. With the rapid advances of artificial intelligence, perhaps the stakes have never been higher. We are witnessing the fastest build up of computation capacity in history; a revolution almost certain to shake the foundations of society – capital vs. labor, human vocation, even what is left of our shared reality – likely necessitating a rewriting of the social contract as we know it.

And, today, the cards are stacked in favor of the Tower.

Today’s cutting edge models are fueled by two primary inputs: compute and data. Due to the tenants of “aggregation theory“, the internet has effectively balkanized into a handful of closed ecosystems, ramparts reinforced by compounding investments in a decade of artificially low rates, which have led to the monopoly cashflows and colossal data sets needed to dominate the age of intelligence. There has perhaps never been a more prime opportunity for power concentration by a small minority; a global economy of consumers, enterprises, and even governments underpinned by a few increasingly all-powerful foundational models, managed by small clusters in commanding heights.

But actually, it’s even weirder. AI IS the network. The LLMs are us. An intelligence trained on the collective output of humanity, staring back at us; the hivemind commandeered into a closed system: the square, bottled up, “trained” and redeployed back to us under the dictates of the tower. For a fee. There is something incredibly… off about the entire market structure.

And yet, the swarm is already assembling in response: a coalition of developers, investors, and users determined on prying AI from the iron grip of Big Tech and painting a vision for user-centric intelligence.

The following report is a layman’s exploration into the AI value-chain: a peek into the awesome resources being deployed by the hyperscalers and the emergent decentralized AI ecosystem, assisted by cryptography & distributed incentives, aiming to present a viable alternative.

Are we heading for a world of a few 10 trillion parameter models governed from four thrones in the Pacific Northwest? Or are we destined for a tapestry of models in the millions underpinned by open source communities and collective ownership? An internet of agents? A Hivemind.

Of history’s countless concentrations and diffusions of power, the next chapter between the Tower and the Square is perhaps it’s most consequential.

Your Size is Not Size

Google, Amazon, Meta, Microsoft, and Apple are currently sitting on over US$400b in cash-like assets (now clipping >5%). Furthermore, their core businesses are throwing off the equivalent of >1.5% of GDP in operating income each year. The race for AI supremacy is just getting started.

The four hyperscalers (above less Apple) have combined to spend US$467b in capex over the last three years…

With recent analyst estimates expecting >$1 trillion in combined outlays over the next five – an absolutely mind-blowing number in capex spend, passing all but 18 countries’ GDP levels. A significant portion of this spend is going straight to the AI server market, dominated by Nvidia.

To put these numbers in perspective, the AI server market is set to eclipse the entire CPU market by ~40% in 2025, US$166b vs. US$120b. The investment in AI datacenter capacity is the largest outlay in the history of compute by a wide margin.

Two scenarios come to mind: 1) the cloud boom of the 2010s which was met with robust demand as companies of all sizes recognized the benefits of outsourcing core IT infrastructure or 2) the US$90b in fiber capacity laid during the telco boom much of which remains unutilized even today.

Looking at the market size for… intelligence… my hunch, and clearly the C-Suite of the hyperscalers, is for these massive outlays to be met with robust demand and – dare I say “oligopoly-esque” – returns. US corporates are sitting at ~14% in cash-like assets and margin levels not seen since the 1980s. Every earnings call and board meeting is rife with existential AI disruption discussions. The demand will come.

Investments in cloud infrastructure started out with negative margins and have since risen into the 60 – 70% gross margin range as the market has consolidated around three primary vendors servicing a Cambrian explosion of applications and cloud transitions.

The setup for GenAI feels very similar.

In a recent piece, Apoorv Agrawal outlines the discrepancy in the GenAI value-capture at different layers in the stack vs. the current state of cloud. The outlays in cloud eventually led to ~US$200b in cloud revenues powering a further ~US$400b revenue app ecosystem on top.

This follows a familiar pattern outlined by Union Square Ventures where infrastructure outlays lay the groundwork for the ensuing application boom.

Source: USV blog.

We are still very much in the infrastructure build out phase of Gen AI with the vast majority of profits to date accruing to Nvidia and the semi layer of the stack, but value capture is likely to shift up substantially over time

Not only cash rich corporate IT budgets but also a significant portion of the ~US$600b in VC dry powder will find their way into AI server spend over time, not to mention AI-focused sovereign funds like those of Saudi Arabia and Abu Dhabi both announcing targets for ~US$100b in AI focused investment vehicles to deploy in the decade ahead.

Like cloud, there is increasing reason to believe much of the middle layer of the Gen AI stack will be dominated by a few major players who have combined the capital, talent and data to hone amazingly capable infrastructure and models which will be rented out: intelligence-as-a-service.

Paradox of the Open Web (Consumer Aggregation)

One of the more prescient paragraphs I have read regarding the evolution of the internet came from an interview with famed American author David Foster Wallace at the tail end of his Infinite Jest book tour in 1996.

“…That Interlace will be this enormous gatekeeper. It will be like sort of the one publishing house from hell. They decide what you get and what you don’t.

Because this idea that the Internet’s gonna become incredibly democratic? I mean, if you’ve spent any time on the Web, you know that it’s not gonna be, because that’s completely overwhelming. There are four trillion bits coming at you, 99 percent of them are shit, and it’s too much work to do triage to decide.

So, it’s very clearly, very soon there’s gonna be an economic niche opening up for gatekeepers. You know? Or, what do you call them, Wells, or various nexes. Not just of interest but of quality. And then things get real interesting. And we will beg for those things to be there. Because otherwise we’re gonna spend 95 percent of our time body-surfing through shit that every joker in his basement—who’s not a pro, like you were talking about last night. I tell you, there’s no single more interesting time to be alive on the planet Earth than in the next twenty years. It’s gonna be—you’re gonna get to watch all of human history played out again real quickly.”

-David Foster Wallace (“Although, of Course, You End Up Becoming Yourself” by David Lipsky)

The internet evolved into propriety applications and ecosystems not out of malice but because the limiting factor is not content but human attention (shout out to @mrink0). Today, real scarcity online is found in effective curation – in sorting through the endless bits to find what is actually relevant to the user. Per Ben Thompson’s aggregation theory, capturing demand onboards its own supply in a world of zero marginal costs.

The truth is that Google and Apple and Meta and Amazon and TikTok and Tencent have succeeded in attracting users because they are expert curators: serving us – out of the trillions of bits on the web – those we find most relevant. And this positioning – as “wells” or “nexes” or “gatekeepers” – has allowed them to extract greater amounts of capital and data – the two assets most essential in our new AI “scaling wars” in the era of Transformers.

We can bitch and moan all day about the App store’s 30% and Google and Facebook’s ad duopoly and Amazon driving mom and pop’s into the dirt – and yes, it’s true that closed networks move from open to extractive over time…

Source: Chris Dixon: “Why web3 matters”

And yet, we use them. We use them because the alternative is chaos: saving me time, protecting against fraud, ensuring seamless payments, surfacing material most relevant to my desires.

Why should we expect AI to end up any differently?

In a way, AI is the ultimate aggregator. Amazon, Google, Booking.com – and potentially TikTok & Reels – all rolled into one giant front-end. The user’s interface to the internet. The best tool – closed or open – which most effectively matches user intents with the desired information or action will win. Convenience has proven time and time again to trump ideals in the marketplace.

Concentration appears set to continue, and it’s not limited to the consumer side.

The Great Bundling (Enterprise Aggregation)

The 2010s saw enterprises and startups alike rush to become cloud-first in their deployments. By outsourcing infrastructure, enterprises found they could scale up dynamically with demand, capitalize on the amortized spend by Big Tech on talent and services, and focus their resources on differentiation at the app layer.

This hardware consolidation is now creeping up the stack with the hyperscalers increasingly taking share from “best of breed” software solutions with their “good-enough” vertically-integrated offerings.

Instead of having five different vendors across your database -> ETL (extract transfer load) -> Business Intelligence / Analytics value-chain, why not simply work with one vendor who does it all?

Artificial intelligence is accelerating this vertical integration of data pipelines by seamlessly pulling data from the database straight into relevant C-Suite dashboards. Why deal with painful integrations between different vendors?

This has been the Microsoft business strategy for decades, leveraging their distribution and footprint within the enterprise to sell an integrated bundle which is greater than the sum-of-the-parts, squeezing out competitors. 

I expect to see a similar playbook unleashed within the lower layers of the AI Stack where integrating hardware and software and data will lead to superior levels of performance vs best of breed solutions.

Between increasing returns to aggregation on the consumer side and increasing vertical-integration and bundling within the enterprise, the concentration of economic activity in only a handful of enterprises is reaching staggering heights.

Source: @zerohedge

While the dictates of Schumpeterian disruption may suggest a surge of startup activity to attack this concentrated value capture, its important to note that, in many ways, AI is a “sustaining” technology as opposed to a fundamentally “disruptive” one.

From The Innovator’s Dilemma by strategy icon Clayton Christensen:

“Most new technologies foster improved product performance. I call these sustaining technologies. Some sustaining technologies can be discontinuous or radical in character, while others are of an incremental nature. What all sustaining technologies have in common is that they improve the performance of established products, along the dimensions of performance that mainstream customers in major markets have historically valued. Most technological advances in a given industry are sustaining in character…

Disruptive technologies bring to a market a very different value proposition that had been available previously. Generally, disruptive technologies underperform established products in mainstream markets. But they have other features that a few fringe (and generally new) customers value. Products based on disruptive technologies are typically cheaper, simpler, smaller, and, frequently, more convenient to use.”

Judging by the stock charts of the MAG 5, the market certainly seems to be viewing AI as a “sustaining” technology. In this world, “Big Data”, “Big Iron”, and “Big Distribution” just keep winning. Economic activity further concentrates and and accessing cutting-edge intelligence will come with increasingly extractive taxes as competitors are pushed out of the market.

In short, a redux of the last decade on hyperdrive.And yet, there are reasons Sama and Satya and other nouveau-Bourbons should not be entirely comfortable up there on their silicon thrones. The wall of GPUs may be high, but the netizens are assembling in front of the Bastille.

Open Rebellion

While the hyperscalers have clear advantages, their defenses are not without vulnerabilities. As Michael Dempsey points out, moats in AI actually seem to decay much faster than their network-effect-enveloped web2 progenitors, with open source competitors nipping at the heels of each new generation of models upon arrival (see leaked Google Memo). If this continues, public market investors may balk at underwriting the continued massive hardware outlays required for each subsequent generation of models as leads are eroded and inference proves a race to the bottom:

Source: Semianalysis

If there is 1) any slow down in scaling or 2) a new model architecture breakthroughs or 3) continued advances in synthetic data generation or 4) advances in distributed training… the balance between centralized and open alternatives may shift.

While powerful, centralized models suffer from a few serious drawbacks:

1. Future competition: many businesses may rightly be wary of lock-in with hyperscalers likely to cannibalize their offerings over time
2. Red Tape: while regulatory requirements such as SB1047 may only serve to lock-in incumbents like with finance and Big Tech, growing safety concerns may also slow down the breakneck progress of current leaders, providing time for open or decentralized alternatives to close the gap.
3. “Censorship” / User backlash: ever-increasing “rule lists” by centralized providers may serve to nerf models and fragment demand towards less “censorious” alternatives like Venice.ai. The narrative of “tech elites” with closed, opaque models underpinning much of our economic activity and serving as the “source of truth” is ripe for populist backlash in an era of growing distrust in elite institutions:

There is a case to be made that even if open source models are worse, mass iteration of apps built on top from developers keen on avoiding lock-in, bias, or heavy regulatory oversight may make open source ecosystems superior.

These ecosystems would also benefit from greater security and transparency. Functioning in the global commons, software is consistently “hardened” by innumerable attack vectors, and there are many use cases which will require portions of the stack to be “transparent” or “provably unbiased” (legal liability around civil rights law or healthcare for instance…).

Incrementally, as smaller models are trained or fine-tuned on higher-quality, more relevant data, they may prove more than enough to release the next wave of apps across different categories – the dynamism of the swarm outcompeting closed ecosystems.

These open source advantages have had a massive shot in the arm from an unlikely source…Having been burned by Apple on ATT and in an eternal struggle with Google for ad dollars, new-Zuck is going “scorched earth” to avoid external dependencies, opening the weights of his deca-billion $$$ training runs (well, at least up to 700m MAUs without a license).

While this combination of Meta’s data and financial resources with the open source community’s creativity and iterative speed is formidable, it also represents a huge dependency.

Zuck is ultimately responsible to Meta shareholders. When push comes to shove, this new love affair with OSS is likely to be sacrificed at the alter of capitalism, not to mention US regulatory dictates.

While I’m tempted and heartened by these arguments, the sad truth is that open source by itself is unlikely to offer a superior solution to undefeated capitalist incentives fueling closed source. In Leopold Aschenbrenner’s phenomenal new piece, he makes a credible case for US$100b and even US$1 trillion dollar clusters coming this decade in route to AGI. These staggering outlays seem insurmountable to all but the hyperscalers and largest cadre of nation-states.

As newly-crowned twitter villain John Luttig of Founders Fund recently pointed out:

1. Meta is “not running a charity” and Zuck will stop “opening-weights” the moment it becomes non-strategic or politically sensitive to do so, undercutting OSS’s leading source of cutting-edge models
2. Open source lacks the feedback loop between production usage and model training, and must foot the bill for incremental training data unlike Big Tech competitors benefitting from their aggregation economies
3. The ROI on training runs in the tens (or hundreds!) of billions is more justifiable for closed models than open source

Zuck and his mountain of H100s will not be there for open source forever and when he ghosts, the rebellion will be at risk. If we want to avoid a world entirely dependent on three companies for cutting-edge intelligence, the square will need incremental fire power.

Open source by itself is not enough.

Will my Lambros Please Stand Up

In my estimation, the distributed incentives in crypto offer the only credible path to injecting open source software with enough fire power to combat the powerful incentives leading to “closed”, extractive intelligence.

Without a tool to coordinate and fund the square, the different factions will get picked off by the Tower’s cannons regiment by regiment. Public market investors will continue piling into the hyperscalers, offering ever cheaper financing in a self-fulfilling flywheel of abundant access to capital → outspend competition → smarter intelligence → raise prices → reward shareholders → more financing. As we have already seen by OpenAI and Anthropic bowing before the thrones of Big Tech, no single startup or corporate can compete on its own. Only a coordinated effort, funded by collective incentives, can pool the computational resources, datasets, and talent needed to compete.

As opposed to AI as a sustaining technology, crypto offers a path to AI as a disruptive one. One which pushes power and intelligence to the fringes of the network as opposed to a compounding concentration at its center.

In the event that crypto achieves its vision, the very shape of economic activity would shift, creating a mesh of intelligences to slowly eat away at the siloed corporates and other industrial-era institutions which currently dominate.

While this vision may seem improbable, many of the early pieces are coalescing. Crypto is fundamentally about networks – networks in payments and financial services, yes, but also in infrastructure. And networks are only really valuable once they reach a certain “tipping point”; an inflection where both sides of the market see enough liquidity to supplant earlier networks. Crypto has not yet reached inflection, but its getting there:This bootstrapping is underway in financial services, siphoning off capital and talent into a new financial system. However, the distributed infrastructure being laid in data storage, networking and compute, may prove even more consequential.

Crypto adoption has been slow amongst users because, in many ways, the interface is purpose built for machine intelligence. My suspicion is AI – specifically natural language interfaces and on-chain agents – will accelerate demand for both permissionless value-exchange and infrastructure, encouraging a reciprocal infusion of supply in response.

This series is an exploration into the different layers of the AI x Crypto stack, how they may be symbiotic, and where are the key bottlenecks which limit viable alternatives relative to what is currently on offer by the hyperscalers. However, in order to make this assessment, we to take a quick detour through ML 101.

Ok, nerds, The AI Stack

(The AI fluent should feel free to skip this section). At a high level, the generative AI stack looks something like this:

Source: a16z. https://a16z.com/who-owns-the-generative-ai-platform/, edits by Delphi Research

And zooming in further on the model segments, something like this:

On the training side, recent advances in foundational models have been driven by hyperscaling of hardware compute which has allowed machine learning practitioners to train larger and larger models on more and more data. In effect, there are three primary drivers:

1. Novel model architectures: transformers, RNNs, Sparse models, etc (and the algorithmic efficiencies we are set to reap – i.e. the “unhobblings” per Aschenbrenner)
2. Efficiency gains in hardware
3. Explosion in available data

Overtime, researchers have learned to submit to the bitter lesson: that “brute scaling”, assisted by Moore’s Law, significantly outperforms other approaches. This has led to a war of ever more compute…

To process ever more data…

This report is not a deep dive into LLMs (those are legion, and I am but a lowly crypto analyst), but a generalist’s attempt to tease out the opportunities in the stack most amenable to complementarity by cryptography and distributed incentives. However, this does require an admittedly basic understanding of LLM mechanics and constraints.

Below is my most concise summary.

Model Training 101

Step 1: Data Assembly

Preparation: Where do you get the data? Can you efficiently scrape the public domain without having IP addresses blocked? Can you guarantee the data hasn’t been poisoned? Can you add high-quality proprietary data sets? Can you avoid sensitive / protected data sets and ensuing law suits? Can you bias the model towards higher quality, higher fidelity datasets? Are your data sets clean and model legible? Do certain data sets require labeling? If so, can I incentivize the relevant parties? Do they have the right expertise? Can other LLMs clean the data?

And so on…

Step 2: Architecture Selection: Transformers, RNNs, MoEs, Sparse Models, CNNs, Hybrids, etc

Step 3: Training Process: unsupervised learning aimed at updating parameters to minimize the loss function.

Tokenization: To start, raw text data is fed into the model. Because computers can only read numbers – not text – the process of “tokenization” or converting units of text into “codes” is essential. These numerical arrays or “vectors” mark a word’s meaning and its relation to other words in the language. Each number in the vector is a “weight” or a “parameter” with the current leading models using trillions (GPT4 rumored to be at ~1.8T.).

Hardware: To run models with trillions of parameters and analyze the relationships between words in systems as varied and complex as human language requires a lot of specialized hardware: supercomputers optimizing a constant interplay between data storage, memory, bandwidth, and compute. That being said, we are already seeing highly capable ~7b parameter models running locally. This race between how much each generation of mega-model leaps ahead vs. how quickly these capabilities diffuse into more commoditized hardware will prove essential to ultimate value-capture.

Training: At a high level, the training data is loaded from storage into memory (in batches, based on RAM size and bandwidth constraints) which is fed into interlinked racks of GPU servers to tease out the optimal relationships between tokens.

To date, the mad scramble for GPUs is symbolic of where the constraint has been – with Nvidia’s revenues and stock chart looking like a RoaringKitty-blessed meme stock. However, many analysts expect bandwidth (essentially the speed at which data can move from storage to memory and loaded into the GPU clusters) to replace compute as a key near-term constraint, likely followed by energy and ultimately data (”data wall” discussed later) as we test the limits of intelligence through “scaling” alone*.*

On the other hand, it’s worth mentioning that scaling limits are currently a result of dense backpropagation (layman translation: effectively comparing predictions to actuals and making many iterative adjustments to limit the error rate which are very computationally intensive). “Backprop” alternative algorithms are an active area of research which may significantly alleviate current resource constraints, changing the balance of power between Big Tech and the field. (Paper – A_Survey_of_Backpropagation-free_Training_For_LLMS for those caffeinated enough to engage further).

Fine Tuning 101

Starting with the foundational model from pre-training, fine-tuning involves further refinements based on task-specific data to make the model more effective and efficient in real-world applications.

There are two broad categories: “full” fine-tuning and “PEFT” (parameter efficient fine-tuning).

“Full” Fine-Tuning

More resource intensive approach that updates all the pre-trained model parameters for a more specified task. It is generally is more performant and provides greater flexibility but requires higher memory and compute resources and can lead to “overfitting”, particularly if the task-specific dataset is small.

PEFT

Parameter efficient fine-tuning emerged as a solution to the high computational costs and memory requirements of full fine-tuning by reducing the number of parameters that need to be updated. This allows for fine-tuning of very large models on smaller devices or with limited resources and allows for modular inputs without needing to retrain the entire model. We may be able to make models “smarter” on the fly by applying fine tunes dynamically based on the question being asked.

This shift has clear ramifications for the competitiveness of open, decentralized alternatives by favoring innovation at the edge vs. stuffing everything into massive, capital intensive training runs.

The speed and creativity of the network should not be under-estimated.

Model Inference 101

Inference is effectively when the trained and tuned model is used to make predictions based on new data. In the era of chatbots, this largely means responding to user queries. While inference is not as compute intensive as training, it is far from free (unless running locally for say…your personal AI), with the costs of running inference on 1 token roughly ~2*Parameters in flops. This means that for a one trillion parameter model the number of flops needed to run inference on a 100 token inquiry with a 100 token output should be roughly (1 trillion x 2 x 200) = 400 trillion flops.

While one query is not super costly for high-end hardware, millions of power users with repeat requests and long context windows can add up quickly. The load shift from training to inference is well underway and effective utilization becomes a big deal in profitable servicing.

Bandwidth is also important for inference because of the speed at which users expect responses for different use cases. Because of concerns over latency, so far training has proven optimal in a centralized supercomputers but inference is less intensive and perhaps more efficiently done closest to the application / relevant end user data.

Vectors (sorry) of Centralization

Astute readers may have already begun teasing out how these mechanics lend themselves to centralization, but this write up by @Nirmaanai does a great job of making it explicit, summarized / edited below for brevity:

Key advantages of the Tower:

• Data Access: large proprietary data sets from dominating the second generation of the internet. Large organizations also have war chests and legal departments to better navigate data partnerships and lawsuits
• GPU Cost & Access: even if you could afford the US$30,000 price tag per H100 (and you would probably need >100,000 of them to be relevant), you would politely be asked to join the back of the queue; existing supply long since locked up in bulk purchases by the hyperscalers

Source: Statistica

Meta’s 350,000 H100s expected by year end 2024 would cost >$10b at current prices with Google rumored to be doing >1,000,000 TPU training runs later this year. These are the type of numbers that price out… literally everybody.

• Latency: my understanding is that because bandwidth is a crucial constraint, widely distributed training runs of competitive performance with centralized clusters are unlikely near term. Centralized superclusters seem poised to keep their lead. However, hyperscalers have had decades to optimize their data centers while decentralized approaches are still young. The fundamental research side should catch up, already showing progress using sparse approaches to training and alternatives to “backprop”. Breakthroughs could change this equation in favor of distributed alternatives over-time
• Economies of scale in inference: The cost of keeping models broadly available is quite high with recent estimates suggesting that keeping the GPUs spinning on GPT4 costs >$1,000,000 per day. Most of the pricing is variable on a per token basis and contingent on the number of parameters (more parameters = more calculations & higher costs) as well as the length of inputs and outputs. Higher utilization and “batching” of responses mean cost advantages relative to those fielding less queries – forcing the laggards to underprice market leaders like GPT4 in an attempt to gain share. Per semianalysis, this is another vote in favor of the market consolidating around a few foundational models as the flywheel between utilization and lower costs serves to pull in more customers with better performance.
• Bundling! Do not underestimate the power of distribution and bundling. “Hey, I know that guy hosts your storage and compute and is embedded in all of your enterprise workflows, but outsource your intelligence layer to me, bro” is going to be… a tough sell to most corporate CIOs and IT departments.

And yet, it’s not all doom and gloom for the upstarts. Often disruption is non-obvious, sneaking in undetected from the fringe when incumbents appear their most invulnerable.

AI x Crypto

Because of the above challenges, the ability for AI x Crypto to compete head-to-head with web2 giants on their own turf – like training cutting-edge models – is likely to remain challenged.

However, this does not mean there cannot be a vibrant decentralized AI ecosystem which caters to different customers and use cases today which may ultimately prevail long-term: subtly laying the ground work for AIs shift from a “sustaining technology” to a fundamentally “disruptive” one.

Just as decentralized finance today cannot compete with its web2 / Wallstreet counterparts directly on performance and cost, it none-the-less provides novel properties like openness, composability, and immutability which unlock new use cases. In the event that performance gets “tolerable enough” to compete with web2 incumbents, these additional properties may prove enough to pull ever more activity to the new stack.

AI development may prove similar, with absolute performance remaining the primary initial consideration, but openness, composability, transparency, and more distributed ownership of the models set to underpin society gaining in importance over time. Too many developers have been burned by platform shifts and monopoly rents to not make this a real consideration for the next generation of the internet.

Again, I believe the key unlock to watch here is wallet adoption. Once enough individuals begin interacting through peer-2-peer networks as opposed to through intermediaries, the shape of economic activity may shift faster than many expect. Future economic activity may look much less:

Business -> Business -> Consumer

and much more…

User -> agent -> smart contract -> agent -> user

So far, we have yet to reach this inflection, but we are certainly en route to a world of networked intelligences. Just as Wechat / AliPay effectively hijacked the Chinese financial system (temporarily) by bootstrapping a network of wallets, crypto is ambitiously attempting to do something similar globally: in digital markets like financial services, but increasingly in physical markets like storage and compute.

To date, bootstrapping this network has been held back by the industry’s nigh-impossible onboarding UX, stale (1930s) regulations, and problematically requiring distribution from one of the very duopolies (mobile OS) it aims to disrupt.

However, as we will see, the arrival of agents on-chain may fundamentally change this equation, accelerating both user-onboarding and the demand for machine-native infrastructure.

If you step back, the status quo is actually quite odd. With a host of intermediaries and closed ecosystems harvesting user data and charging monopolistic taxes, both used to invest in a monopoly on intelligence to further consolidate market power, protected by legal frameworks devised in an era where the technology did not yet exist to provide users with an alternative.

If the Crypto x AI vision comes to fruition, we may suddenly find ourselves in a post-inflection world. Where individuals and encrypted meshes serve as increasingly important data repositories relative to larger enterprises. Where more sensitive user data is stored and permissioned locally. Where crypto incentives prove successful in capitalizing a network of storage and compute and developer talent to combat today’s commanding heights: met with an-NLP-enhanced user experience and personalized AIs and emergent ecosystems of agents on the demand side. This demand will not be subject to the same regulatory constraints and need for human legibility; it would choose open and permissionless payment rails and infrastructure. Users’ direct interactions with the network would be intermediated by agents and interact with a growing number of autonomous ones. The growing demand would reinforce the supply and so forth.

But this vision, if it comes to fruition at all, would take time.

In my opinion, the impact of AI coming to crypto will be immediate while the impact of crypto on AI – requiring the buildout of this multi-sided network – will take longer. However, if realized, the impact would be more transformative than many realize.

The AI x Crypto Stack

At a high level, the Crypto x AI stack looks something like this:

And the flows between the various players (pinch-to-zoom assisted for those over 30 I assume), looks something like this:

The landscape here is as complex as it is rapidly evolving. Below is a mere sampling of the types of projects found in each layer: infrastructure (the supply side), middleware (tooling and coordination) and applications (the demand side) as well as my initial take on some of the key opportunities / challenges emerging from each layer.

Infrastructure

To date, the primary crypto-related unlocks in AI x Crypto-related infrastructure are DePIN related: using distributed incentives to coordinate latent supply – across data, storage and compute – in the long tail, accessible in a trustless, permissionless way.

Decentralized compute: projects focused on matching generalized GPU resources – Akash and Render – or specialized GPU hardware – Bittensor and Gensyn – or aggregators coordinating across both camps as well as second-tier cloud providers (io.net) to match the rising demand for AI training, tuning and inference

Decentralized storage: distributed file sharing and storage systems like Filecoin and Arweave focused on permissionless, persistent, verifiable, cost effective storage tapping underutilized supply

Decentralized Data Sourcing & Management: a combination of decentralized data scraping DAOs (Grass) as well as labeling (Lightworks) and marketplaces (Ocean Protocol) etc.

TEE: and other hardware to ensure data and model confidentiality and integrity

Pondering’s first glance hot takes:

GPU Marketplaces:

Looking at the 100%+ CAGR in the AI server market, the near-term demand for compute appears almost unbounded.

While hyperscalers will win the lion’s share of that demand, the market is huge. There remain a large number of under-utilized GPUs in legacy datacenters, not to mention former crypto miners (by some estimates ~9m GPUs were freed up after Ethereum’s merge) which have nothing to lose by matching with less-performant-sensitive demand, gaining share with highly cost competitive offerings.

However, a few challenges are worth noting:

1. Distributed training performance:

Especially on non-uniform, legacy GPUs is a challenging orchestration problem and suffers from latency between clusters, meaning performance will continue to lag hyperscaler offerings, suggesting this supply will mostly target inference (i.e. pinging deployed models vs. training new one’s, expected to be a larger market anyway). However, it’s important to note that breakthroughs on the fundamental research side or GPU ASIC equivalents may impact the magnitude of this disparity.

1. Competition from tier 2 clouds:

Hyperscalers are not the only game in town, with secondary cloud offerings quite cost competitive, though this could end up being a classic case of “co-opetition” between tier-2 clouds and aggregators like io.net

1. Limited proof points of DePIN capturing demand, to date:

Despite crypto-incentives proving a powerful tool for building supply-side capacity, we have yet to see use cases where demand truly inflects. DePin Ninja lists >1100 projects valued at ~US$54b and a measly US$15m in ARR.

The reasons for this are multi-fold:

1. Many DePIN projects have only launched very recently and are still building out the supply side
2. Marketplaces take a long time for flywheels to form, but can get much larger than most expect after inflection
3. Legal / Regulatory constraints on risk averse enterprises

Crypto folks have traditionally under-estimated point three as a barrier to mainstream adoption for protocols. Filecoin, the DePIN OG in distributed storage has been around since 2014 and has yet to find meaningful product market fit despite favorable pricing vs. many enterprise offerings.Source: Coingecko.com, Nirmaan.ai

Yes, it’s partially because they have less consistent pricing / performance guarantees and less tooling and integrations, but, in many respects, its due to legal and regulatory hurdles. Today, most of the demand for storage and compute comes from risk averse enterprises who, subject to compliance in the jurisdictions in which they operate (with regulations often crafted decades ago), require the standard SLAs and counterparties and contracts and arbitration courts to feel comfortable doing business. They are more comfortable with legal contracts than smart contracts. These institutions were crafted for a world of human legibility, not encryption and networked intelligences.

In other words, demand is throttled because, to date, data on users resides primarily in siloed intermediaries, accumulated before proper tools enabling an individual, network / agent based economy could take hold. These siloes and legacy institutions reinforce one another.

While GPUs are much less commoditized than storage, GPU marketplaces are running a similar playbook and may run into issues in consistency, tooling, integrations and regulatory concerns compared with centralized offerings.

I want to be clear that I am a big believer in the potential of DePIN and the signs of life creeping back into the category are encouraging, but near-term I expect GPU marketplaces to remain more of a niche market. However, a niche participant in a market growing at a >100% CAGR and sized in the hundreds of billions can still be a large business.

That being said, the arrival of on-chain agents may cause me to revise this take up considerably.

Data Aggregation / Mgmt

There is a real chance data proves to be the binding constraint on the current architecture de jour’s performance. This, in my opinion, represents one of the largest long-term opportunities for web3 while simultaneously suffering from substantial near-term headwinds: the familiar “cornering of proprietary supply” combined with regulatory headwinds and challenging incentives for coordination on the enterprise side.

Data is not really like oil, “data is like sand“: only becoming valuable at Singapore-importation-quantities – with few ecosystems possessing needle-moving amounts outside of what is increasingly commoditized from the public domain.

Think about your relationship with Google: gmail, maps, docs, search history, calendar, chrome, youtube, video chats… that is an incredible advantage in a world where the software of our new probabilistic computers is data-driven.

Any single company trying to go head to head with that juggernaut will get rocked. A new approach is essential.

“DataDAOs” are one solution to this collective goods problem: using distributed incentives to scrape and format and label valuable training data-sets to more effectively train and tune open source models. These can be monetized via arrangements with larger foundational models or to train / tune highly task-specific models.

Consumer-focused collectives seem to be making more progress. Projects like Grass have a syndicate over >2m strong, tapping residential IP addresses which can skate passed website blockers, scraping the global internet commons with unused CPU cycles. Projects like Lightworks further leverage distributed incentives to match AI and manual labor to optimize these incoming data sets into model ready formats. Incrementally, distributed data collection via robots, smart pins, and other wearables may prove to be a big area and potentially ripe for crypto incentives. Hivemapper is a good example. These are the types of networks that will need to scale massively for dataDAOs to be competitive. Again, this use case depends on a network of near-ubiquitous web3 wallets which enable sharing value with the user.

The opportunity is huge, but challenges remain:

1. Even just defining what is “your data” is exceptionally difficult. The value of data is in the web of relationships between you and the others that you interact with in aggregate which is very difficult to harvest outside of the closed ecosystem which encapsulates all of those relationships.
2. Data marketplaces are one way to solve this problem. However, centralized marketplaces are already large businesses with existing network effects and proven compliance regimes traditional enterprises value, leaving less room for decentralized alternatives.
3. Incrementally, data sharing suffers from mal-incentives with many valuable data-sets better monetized through propriety models or simply a high-priced licensing deal with a Big Tech co.

These challenges are real, but they may not be insurmountable. Outside of data collectives, several other factors like advances in fully-homomorphic-encryption, more performant models tuned on smaller, higher-quality data sets with real-time context, synthetic data generation, and brands finally realizing the cost of their disintermediation by Big Tech, all promise to shift power back towards the Square over time.

While having the most near-term challenges vs. incumbent models, I continue to believe the infrastructure layer for decentralized AI presents the largest long-term opportunity for truly ambitious entrepreneurs backed by aligned communities.

(If you are a founder who disagrees with these initial takes and would like to educate me ahead of next month’s deep dive on the infra layer, please reach out @PonderingDurian on Twitter / Farcaster. Truth is best converged upon one public dunking at a time 🙂 )

Middleware

Within the middleware layer, we have several particularly relevant clusters worth calling out:

• Open Source Models: Llama3, Mistral, Hugging Face, etc – which often form the basis for continued experimentation given the capital demands of training new models from scratch
• Verifiable Compute: using zero-knowledge and optimistic proofs and other tools to ensure privacy, data-integrity and verifiable training and inference
• Coordination Networks: meta-intelligences or algorithms which route intents through the most efficient path to execution
• Developer Tooling: tooling and SDKs required to build out AI applications

Of the group, Verifiable Compute and Coordination Networks strike me as sweet spots to highlight for crypto-based solutions.

Verifiable Compute

For a deep dive on verifiable compute, please check out Dmitriy Berenzon’s excellent post which outlines the core use cases and project landscape:

Zero-knowledge proofs (ZKPs) have long excited the crypto imagination with applications in both privacy and scaling, but are increasingly seen as a core compliments in verifying inputs and outputs from often opaque AI models.

Many use cases, particularly in regulated industries like finance, healthcare, government and education are likely to push towards open stacks due to transparency and compliance. Incrementally, in high-value, lower-trust environments like blockchains, verified compute will be increasingly essential for integrations of off-chain data sets and computation.

Two use cases outlined in the piece are particularly relevant:

1. Data Integrity: to “prove that some piece of data from either onchain or offchain sources, as well as any additional computation on that data, is accurate and has not been tampered with” which enables “verifiable oracles” and “verifiable databases” which give confidence to external data either coming on-chain or used in regulated environments.
2. Machine Learning: zkML (or more cost-effective, competing solutions like opML and TeeML) can be used to “delegate the execution of machine learning models to service providers and obtain proofs that a particular trained model produces the correct output given some inputs (i.e. verifiable inference), which ensures transparency and prevents model substitutions.” Similar arguments can be made in favor of more transparent training – proving particular datasets and algorithms were used, ensuring the absence of bias (highly relevant for many compliance use cases / avoidance of lawsuits).

While zkML is still prohibitively expensive in many cases, high-value use cases like those impacting on-chain transactions or as required by regulation strike me as likely areas for zkML (or competing techniques) to play an essential role.

While there is a world where hardware and open source models become so efficient that many can simply run locally, I am willing to bet verified compute will have a monster decade alongside the proliferation of AI.

Coordination Networks

Coordination or “orchestration” networks are systems or algorithms or “meta-agents” which decide when to apply a given model to a particular intent or query. Incrementally, they can be used when stringing together a series of actions – through different agents or smart contracts – in order to reach a desired goal in an optimal manner.

To date, Bittensor is perhaps the most notable, though not a “pure-play”, example. Instead of carving out a niche as “orchestrator” within the AI stack, Bittensor takes a federated, yet integrated approach to “full-stack” AI development. An ecosystem of dozens of “subnets” where resource providers – of compute, data collection / storage, ML talent – compete to meet the aims of a particular subnet owner. Rewards are allocated by community vote as to the distribution between subnets which are further dispersed based on competitive performance.

Nous subnet 9 is a relevant example focused on pre-training models. Models are submitted and ranked based on who can minimize losses relative to a specific benchmark. This particular instance is problematic because the ranker has to have a view of “the ideal” before it can rank the outputs of other models (which may be better than the benchmark). Even so, we can see the outline of an “orchestrator” presiding over a marketplace of models – a gladiatorial arena of improvements in pursuit of the reward. We could also envision “meta-algorithms” that collect inputs from several different subnets in route to a more complex goal.

This piece from @IridiumEagle provides an in-depth diagnosis of the issues facing Bittensor, but many are already on the look out for what future iterations may look like.

Another example is the Smart Agent Protocol’s “PromptRank algorithm” outlined by David Johnston which aims to create a graph of the relationships between LLMs / Agents & Smart Contracts where “the output is a ranked list of Smart Contracts that a Smart Agent can interact with based on the user’s choice”.

As we have discussed, curation is increasingly essential in a world of infinite options, and open alternatives can help avoid a default to curation by closed models.

The Smart Agent Protocol vision is ambitious, pushing towards a more user-centric version of the web based on tethering smart agents with locally stored user data, and external actions permissioned by signature from a web3 wallet, routing users’ intents through external smart contracts, ranked by safety, privacy, openness, efficiency etc. to most effectively accomplish a user’s goal.

Source: Smart Agent Protocol Community Paper, David Johnston

The network would be made up of a web3 wallet, an LLM trained on web3 data, a ranking algorithm to score the optimal smart contracts for the user, and long-term memory of user data stored locally or in a decentralized cloud to provide broader context to smart agent actions. This vision has recently come to life with the launch of Morpheus – an open “network designed to incentivize the first peer-to-peer network of personal general purpose AIs that can execute Smart Contracts on behalf of a user”. Since accepting deposits in Feb, MOR has already attracted >139,000 staked ETH (>US$500m at current prices) with staking rewards used to fund initial ecosystem development.

The demand for a user-centric web may be fringe today, but it is gaining steam.

App Layer

We can’t really forecast what will emerge here, but the below categories are worthy of attention.

• Personalized Agents – from consumer layers like myshell.ai connecting users, creators, and open-source researchers to personalized AIs like Venice.ai
• “Agentic protocols” – effectively extremely low headcount, largely autonomous on-chain enterprises which combine agents and crypto rails to outcompete traditional businesses
• 10x improved UX – soon gone will be the days of humans interacting with unwieldy wallets and smart contracts; agents will emerge as the new front-end
• **DeSoc: Identity, Privacy, Governance** similar to the unlock envisioned by Morpheus, DeSoc envisions users as opposed to corporates as first class citizens online
• Automated smart contract auditing – solutions like testmachine.ai should significantly increase shipping speeds. Done are the days of a 3 month wait period for for a top-tier audit
• And much more…

Again, I will save the deep dives for future reports, but the essential unlock here is two-fold:

1. UX Improvements: a substantially enhanced end-user experience enabling on-chain interactions through natural language. This should do wonders in onboarding the next billion users, helping to propel wallet adoption towards the saturation levels needed to unlock many “real-world” use cases. As my colleague Luke outlined in a recent post, perhaps the universal front-end for protocols will simply become natural language:

You: “I have 10,000 USDC. Get me the best low risk yield you can find. I need instant access to it, so don’t lock it up for more than a week”

Agent: No problem. I recommend an initial distribution of 50% in Aave, 20% in Ethena, 20% in X and 10% in Y”.

You: “Sounds good. Do it”

The legibility on both sides (human and machine) will be difficult for legacy stacks and institutions to replicate.

But why stop at financial services?

1. Agentic Protocols: First coined by GoodAlexander in this piece, he defines Agentic Protocols as a “self-developing AI driven entity that aims to have no human employees. It generates cash flow by licensing IP or other technology products, facilitates network economics or speculates. It has its own native cryptocurrency token which 1) is the currency by which its products are sold or licensed which 2) allows human or AI users to participate in the upside of its financial activities and 3) validates and pays for the opex of the system which is primarily compute, training, and storage”.

I am less cynical than Alexander regarding the role for humans, but he is directionally correct. Agents will only get more capable ****and on-chain rails enable machine-accessible storage, compute and talent networks which smart agents can tap themselves – or through orchestrated networks – to achieve ever more complicated, long-term goals.Figure 1. Left: a concept timeline of AI evolution with increasing performance. Right: block diagram of activities for humans and different forms of AI.

Source: Davide Crapis: The Internet of Agents

In this view of the world, consumers begin interfacing less with traditional businesses and more with agents: economies moving from B→B→C to economies of networked intelligences.

This shift in demand would inevitably impact where supply gets sourced with big implications for protocol accessed storage, compute, and intelligence relative to closed APIs. If we indeed expect future economic activity to be much more agent driven, on-chain utilization may inflect much faster than many expect.

If the hyperscalers expect a return on US$1 trillion in capex from corporates, then what should distributed infrastructure expect from agents?

The low DePIN utilization rates today may prove to be temporary indeed.

The Square is Filling Up

It’s undeniable that the data, compute, and distribution moats of Big Tech are very deep. These moats are reinforced by nation-state levels of investment and backed by political systems often captive to corporate interests and legal systems hostile to decentralized alternatives.

The logical path on Wallstreet is to expect more of same: a rerun of the 2010s on hyperspeed and a further concentration of power and profits in a small minority of large enterprises set to own super-human intelligence.

The market is positioning for a further consolidation by the Tower…

And yet, activity is emerging in the plaza below. A growing band of misfits from disparate corners of the web realizing this is not the future they want. That, in fact, we can craft open intelligences, collectively, to rival those of Big Tech. That the pooling of talent, compute, and data from billions of individual nodes can compete with even the most powerful incumbents.

Bitcoin is proving this with money. Who is to say we can’t do it with intelligence?

**

The Tower may be tall, but one by one…The square is filling up.

**

A big thank you to @IridiumEagle from Ambient for letting me tap his AI knowledge in this report. Any nuance shown is undoubtedly his; any errors are certainly my own.

Additionally, while they may not agree with all of my takes above, I would like to thank @Shaughnessy119, @lukedelphi, @mrink0, @ceterispar1bus, and @CannnGurel at Delphi for their feedback and internal debates as well as the numerous members of the ex-Machina Crypto x AI TG Chat for helping me, a normie, get up to speed as fast as possible.

*@PonderingDurian, signing off.