Join Delphi Research today and immediately get access to our full Member Portal!
Join Delphi Research today and immediately get access to our full Member Portal!
Back

Flashing A Light On The Dark Forest – Part 1

Oct 13, 2021 · 13 min read

By Joo Kian, Genevieve Yeoh, Paul Burlage, and Medio Demarco

The concept of Maximal Extractable Value (MEV) has been thrust into the spotlight over the past few years, particularly as DeFi has emerged, becoming an integral component of a network’s economic incentives. A simple definition is that MEV is the maximal value that can be extracted from deciding the transaction order in a block. The game theory around MEV, however, is far from being so simple.

Importantly, MEV has negative externalities at both the network and user level. Regarding the former, it can lead to gas bidding wars and reverted transactions as bots increase the amount of gas they’re willing to pay in an attempt to front run the transactions of unknowing users. Regarding the latter, it’s these users who pay the price when those bots succeed, both in the form of higher gas fees and having their trade front run. If you don’t think MEV has ever disadvantaged you, take a look at http://sandwiched.wtf to see for yourself. 

Given the scope of the MEV problem, there are several solutions being worked on that seek to mitigate it via different means. The focus of these solutions ranges from innovating at the base-layer’s consensus design, creating new middleware infrastructure, and even retooling at the application-level. Which solutions gain traction and how they evolve over time will impact every user, developer, miner and validator of these networks.

In this two-part report series on MEV, we’ll primarily focus on Flashbots, the leading MEV mitigation tool that most people underappreciate, and CowSwap, a promising newcomer on the DEX aggregation scene with potential MEV benefits. We’ll start by dissecting how MEV works before moving on to a Flashbots deep dive. Once that foundation is established, we’ll continue building on it with CowSwap in Part 2. Join us as we adventure into the dark forest of MEV….


Understanding MEV

Before we discuss the intricacies of MEV, let’s review some blockchain basics to better contextualize it and perhaps even clear up some misunderstandings:

  1. When a user signs a transaction, it is broadcast and ends up in a public mempool run by a node, alongside a list of other unconfirmed transactions.
  2. While those transactions sit pending, Miners race to solve the complex hashing puzzle first (this is why they need all that beefy hardware).
  3. The Miner who wins the race earns the block reward, which is derived from gas fees & new issuance, and gets to order the transactions in the next block.
  4. The winning Miner looks at the pending transactions in the public mempool and starts ordering them to construct a block. Importantly, the order is not based on time but rather incentives.
  5. Ideally, those incentives are gas fee based. The more a user is willing to pay for their transaction to be confirmed in the next block, the more priority the winning Miner should give it when constructing a block.
  6. Once the winning Miner has constructed a block, they share it with the other nodes on the network who check its validity and then add it to the blockchain.

Simple enough, right? Not exactly. Recall that the transaction order is entirely based on incentives. While the first incentive is the amount of gas paid, the second emerges from exploiting front-running / arbitrage opportunities (i.e. MEV). Importantly, it’s the mere existence of the second incentive that can cause the first one (i.e. gas paid) to spike. Why? Because miners aren’t the only actors at play here, unaffiliated arbitrage bots also exist.

There are three primary ways that bots try to extract MEV, as shown in the diagram above. The first is Front Running, where a bot sees a pending transaction in the mempool and pays higher gas to jump in front of it, often causing the affected transaction to fail in the process. This would be done to buy or sell a token before the other person, for example. Next we have Back Running, where a bot detects that a target transaction will cause a large mispricing of an asset and immediately clears the arbitrage opportunity to bring the prices back to normal after. Lastly, we have Sandwich Attacks which are essentially a combination of both. Basically the bot front runs you before your trade drives up the price, then immediately sells for a risk free profit right after (slippage plays a big role here). Sandwich Attacks pose a problem for big traders as they have to split up their orders into smaller blocks for execution to prevent being attacked.

Given those dynamics, it should be unsurprising to learn that DEXs are most affected by MEV, as shown in the charts below.

Now that we’ve covered the different types of MEV, let’s walk through how they would work with an example. Imagine the average gas fee is $1. An arbitrage bot monitoring the mempool identifies an opportunity to execute a Sandwich Attack in the next block and earn a $1,000 profit. However, the transaction order of that next block is not within their power to decide, it’s up to whoever the winning miner is. So, what does that bot do in this scenario? The only thing it can do, incentivize miners to prioritize its transaction by paying a higher gas fee. Of course, this bot doesn’t exist in a vacuum either. There may be several bots all trying to exploit the same opportunity who are also willing to pay a higher gas fee, thus kicking off a bidding war between the bots. If the average gas fee is $1 and there is a $1,000 profit opportunity on the line, the arb bots can basically spend up to that $999 max margin on gas before the situation becomes unprofitable. Note that only one of these bots can win.

This example demonstrates three key points:

  1. While the winning miner could exploit the MEV opportunity on their own, they don’t need to in order to boost their earnings. They can operate business as usual, let an arbitrage bot go through the hassle of identifying / bidding for MEV opportunities and then collect the boosted gas fee they paid as a result. The relationship between miners and bots can thus be synergistic rather than adversarial.
  2. When a bot is searching for pending transactions to front run, it does so by observing the public mempool.
  3. The bots that don’t win the opportunity have their failed bids revert on-chain which results in artificial blockspace scarcity.

Keep these points in mind because they’ll become relevant again in the next section.

Flashbots – Auctioning MEV

Since its launch, Flashbots has gained impressive traction. On Ethereum, ~93% of the total hashrate has adopted Flashbots, which is measured by aggregating the individual hashrate of each mining pool that runs MEV-Geth and receives transaction bundles. Currently, ~54% of all blocks include transaction bundles from Flashbots. By participating in Flashbots, current miners have seen their revenue increase by 3.3% on average over the past 30 days. 

Flashbots, in their own words, is a research and development organization formed to democratize the profits and mitigate the negative externalities and existential risks posed by MEV. Ok, that sounds good …. but what exactly is it? We’ll sync up our Flashbots explanation below to the pertinent key points from the previous section:

  1. It brings together the various miners and bot operators (i.e. “Searchers”), giving them a more formal way to cooperate and share in the upside together. In this model, miners effectively outsource the work of finding the optimal block construction for MEV extraction.
  2. Searchers watch the public mempool for transactions that create MEV opportunities. They then bundle those transactions in an optimal way and submit those bundles to a private communication channel alongside a sealed bid. The miner that wins the next block chooses the best bid, takes the bundle from the private pool and uses it to construct the block. Users/developers can also bypass the public mempool and submit transactions/bundles directly to Flashbots for pre-trade privacy via this system (i.e. Flashbots Protect).
  3. Because the MEV bidding process is done via Flashbots’ private channel, it reduces the negative externalities tied to gas bidding wars and failed transaction reverts.

The diagram below illustrates a simplification of Flashbots’ Auction architecture. Before we run through the steps, let’s first define the participants, of which there are three – 1) Searchers, 2) Relayers and 3) Miners. Searchers are generally bots that scan the public mempool for MEV opportunities, although they can also be regular users. Relayers operate the private channel shown below and propagate transaction bundles. Notably, we left Relayers out of the diagram because that component is currently centralized, for reasons explained at the bottom of the Relayer section here. However, Flashbots is working to decentralize this component. Finally, we have Miners whom require no introduction.

In the first step, shown above, Searchers monitor the public mempool for MEV opportunities. (#2) Once identified, they take a targeted transaction and bundle it with their own in a way that extracts MEV. Since the Searchers are competing against each other for MEV opportunities, (#3) they each attach a private bid to their respective bundles before sending them to Flashbots’ private channel where everything is shared. The bid can either be expressed via gas price or a direct payment to the miner. (#4) Afterward, the winning miner for the next block selects the best bundles from the private channel. Importantly, and this part can be tricky, miners don’t prioritize transactions by profit sharing % but rather by payment per unit of gas, as Luke Van Seters describes here. Finally, (#5) the winning miner constructs the next block, pulling from both the public mempool and private Flashbots channel.

Now, if you’re an arbitrage bot the explanation above probably sounded really lucrative to you. However, if you’re just a boring human that regularly uses Ethereum, you might be asking yourself – “how does Flashbots help me directly though?” With this in mind, let’s talk about Flashbots Protect, whose public beta was recently unveiled on October 6th. To best explain what it is, we’ll build on the previous diagram, as seen below. The main difference is that Protect makes it easier for regular users to submit their transactions directly to Flashbots’ private communication channel, bypassing the public mempool in the process. The primary benefit here is the pre-transaction privacy it enables, which hides your trade from the frontrunning bots stalking the public mempool. Miners are also supposed to give transactions that are submitted this way priority at the top of blocks. What’s enforcing that? Importantly, miners are expected to follow the rules when participating in Flashbots or else they could be excluded from participation moving forward if caught doing something off limits. 

We should emphasize that Flashbots Protect is still very new and has limited public documentation so far. Due to this, the following includes our own speculation and open questions. At the surface level, it seems a bit curious why a tool that facilitates MEV extraction would give people a way to bypass it entirely. For example, imagine an extreme scenario where everyone started using Flashbots Protect, thus eliminating MEV in its entirety. While that’s completely unrealistic, in such a scenario, the miners and bots who had historically benefitted from MEV would be a victim of Flashbots’ success, and likely disengage. All this raises the real question – Are all of the transactions submitted via Flashbots Protect safe from MEV, or are certain types still possible? For example, front running protection is clearly specified but what about back running? To get those answers we’ll need to wait and see.

As the previous section showcases well, the thing that can be so confusing for people about Flashbots is that its existence is neither entirely positive nor negative. Does it completely eliminate MEV? No. Does it minimize MEV? Kind of. But does it mitigate the negative externalities that arise from its extraction? Yes. This is evident in the chart below when measuring 1) – the decline in the Avg % of Gas Used for MEV extraction and 2) a reduction in reverted MEV transactions.

There is a lot of ongoing experimentation targeted towards MEV protection. There are applications building on top of Flashbots, in addition to others looking to compete with it. An example of the former is mistX, which is a DEX front end that essentially makes it even easier for people to utilize Flashbots Protect. In the competitor category, you have Eden Network. A full review of Eden is beyond the scope of this post but there are valid criticisms to be raised about the viability and sustainability of its incentive alignment.

Closing Thoughts

Flashbots may not be the perfect MEV solution, but it’s doing what it can given the hand we, as users, have all been dealt. It has helped improve the situation, at least with regards to mitigating the negative externalities from MEV, and it should be said that Flashbots Protect has a lot of promising potential. Overall, its subtle benefits have likely helped you at some point in time without you realizing.

An important development worth highlighting on this front is that Ethermine, who as the name implies is an Ethereum miner, recently decided to prohibit DEX frontrunning on blocks they mine. In traditional finance, frontrunning trades is usually not allowed and heavily regulated, so from a compliance perspective we can understand the logic behind not wanting to facilitate, or at least profit from, it. However, the knock on effects of a miner making this decision is the crypto equivalent of opening Pandora’s Box. What about censorship resistance? Should miners be liable for other unsavory actions they process when constructing a block? These are the hard questions and they likely won’t be answered anytime soon.

Ultimately, the problem of MEV stems from the fact that miners aren’t incentivized to provide the best, most fair execution when ordering transactions. As a miner, there is no on-chain penalty from exploiting MEV, only upside to be had. MEV and its negative externalities only exist because of how sub-optimal that incentive structure is at the network level. But what if we flipped that script on its head? What if, as a miner, winning the next block reward partially required submitting proof that the transaction order you constructed gave the users best execution? Provide the optimal solution, win the next block. This is a neat thought but it is riddled with many holes that make incorporating something like it at the network level challenging in the best case, and infeasible in the worst.

However, that same idea of rewarding an entity that constructs the optimal transaction order on behalf of the users is alive elsewhere. In our next post of this series, we’ll cover CowSwap, a DEX aggregator with a unique twist – offering MEV protection.