Block scoring for mev-boost relays

Relays
,
1

We need to come up with a standard way to score the value of a block under the mev-boost architecture. This standard is a dependency for a multitude of components including relay monitoring, validator accounting, builder payments, block explorers, payment proofs, and mev hiding. Defining this early and correctly will help avoid a lot of pain and confusion in the future.

Note that a “block score” is not meant to be a formal definition of realized extractable value since this is a difficult metric to quantify. Rather, a block score is the standard metric recognized by the mev-boost system and used for profit switching between relays. An example of a validator payment which would clearly fall outside of the scope of a block score is a transfer to the validator account on an L2.

Some definitions

For clarity, lets refer to the feeRecipient field set by the block builder in the block header as the coinbase address. We can refer to the address provided by the validator to collect fees as the fee recipient. The builder is not required to set the fee recipient to the coinbase address on the blocks they produce.

For context, here are the various payments methods on Ethereum which are relevant to keep in mind in this discussion:

  • base_fee: amount defined by protocol, always paid from user → network, user can define upper bound they are willing to pay using maxFeePerGas, the user needs to have sufficient ETH balance in their account before transaction execution - this is not MEV since there is no way for the validator to get this payment and therefore should not be included in the block score. The payment cannot be made within of EVM execution (unfortunately).
  • priority_fee: amount defined by the user as maxPriorityFeePerGas, always paid from user → coinbase address. The user needs to have sufficient ETH balance in their account before transaction execution. The priority fee is debited from the user account and credited to the fee recipient at the end of transaction execution.
  • block.coinbase transfer: the EVM has an opcode which allows end user to send funds directly to the coinbase address. The recipient of this payment is the same as the priority_fee payment recipient, but this transfer triggers EVM execution including associated gas costs and security risks.
  • regular transfer: anyone can create a transaction which sends eth to any other Ethereum account. Since the proposer is publishing their desired fee recipient address to the public, it’s possible for any user to send payments directly to this account.

Some options

1. Block level scoring

Block level scoring is the difference in the balance of the fee recipient account before and after the block execution.

This is probably the most simple, intuitive, and generalized way to score a block. The main downside I see here is that a validator cannot withdraw fees in the same block they propose without impacting the block score.

Constructing a payment proof for this scoring method requires a merkle proof on the balance of the fee recipient in block n - 1 and a merkle proof on the balance in block n.

2. Transaction output scoring

Transaction output scoring is the difference in the balance of the fee recipient account before and after the execution of the last transaction in the block.

This is more restrictive than the block level scoring as it does not allow for the fee recpient address to be set as the coinbase address. It instead requires the builder to set their own address as the coinbase address and to include a transaction payment transaction in the block.

While it solves the withdrawl problem, it adds some gas overhead to every block and makes payment proofs non-trivial and maybe even impossible (since you can prove that the transaction was included, but not that it did any particular state transition).

3. Transaction input scoring

A slight modification to transaction level scoring could be to score based on the transaction inputs rather than the output. Specifically, the payment transaction can be required to be initiated from an EOA, to have the payment in the value field, and the fee recipient address in the to field of the transaction.

One advantage of this approach is that it allows for easily creating transaction inclusion proofs as well as querying the block score without needing to simulate the block (assuming the transaction did not revert).

4. Event level scoring

Event level scoring means looking at a standard event in a given transaction. For example:

event BlockScore(address builder, address validator, uint256 value);

This approach shares some of the benefits of transaction input scoring, but adds significant complexity and requires predefined conventions to determine which transaction in the block and which event index should be used.

Conclusion

Given the options outlined above, we are leaning towards the block level scoring as it is the most intuitive and least restrictive scoring method and provides a very specific definition to the block value. While it makes validator accounting slightly more difficult due to not being able to withdraw in their own blocks, it should not have a big impact beyond the possibility of delaying withdraw transactions by a few blocks and therefore is an acceptable tradeoff.

There are likely other options and tradeoffs I have not yet thought about and would like for other ideas to be submitted.

9 Likes
2

We also agree on Block level scoring as the suggested solution.

However, we will be open to any other solution taking the below into consideration:

  1. Easy to implement as a builder
  2. Easy to verify as a relay supporting many external builders
  3. In the case builder takes no fee - we prefer not to add an extra transaction and save on gas
3

Block level scoring is by far the simplest approach. Regarding the downside: there are various mitigations, such as having a more advanced scorer that ignores transactions with a reduction in balance or letting block builders know that including transactions that reduce the balance of the fee recipient will harm their chances of the block being picked up.

4

Ignoring transactions which reduce the balance seems wrong as it opens up the ability for a validator to inflate their block scores through reentrancy.

Assuming validators send withdrawal transactions through the tx pool, they should naturally never land in their own blocks as builders would filter those transactions out in their block value optimization. If 30% of the network uses the same fee recipient, then they could experience slower withdrawals, but not significantly.

5

Hey, Flashbots community!

Eugene M. here from Lido protocol devteam.

We definitely support the block scoring effort since it’s crucial to maintain proper monitoring and assessments.

Block-level scoring looks prominent, however, I can confirm that our architecture presumes huge daily withdrawal of accrued fees from the feeRecipient address (which is the same Vault SC for all Node Operators participating in the Lido protocol). As far as I understand, that block would have been considered with negative score and proposer would have to fallback to mempool if had treated without additional mitigations.

4 Likes
6

Does it? From my understanding, transfer of value to another address is atomic so there is no re-entrancy that can be applied. If a trace of transaction x shows a balance increment to address y then that is the sum of all transfers to the address. Those transfers can only be increments, unless the transaction is from y or y is a smart contract that allows third-party withdrawals (which seems unlikely). If the thought is that this could apply to ERC-20 tokens, then again that’s only true if the sender has a transfer allowance for the fee recipient’s tokens (which seems very unlikely).

7

I am thinking of the case of setting fee recipient as a contract that transfers out and then back in in a loop.

Since validator can define any fee recipient they want, they can do this just to troll if they want.

I don’t immediately see a simple rule that filters for this.

8

Perhaps I’m confused here, but there are two values floating around for a block:

  1. the value to the builder
  2. the value to the proposer

If you are calculating 1 then there is no benefit to a builder to include transactions that inflate the value, as it would give the impression that the builder is keeping a larger portion of the MEV for themselves. If you are calculating 2 then re-entrancy doesn’t apply as what matters is the delta at the end of the transaction, not each movement of funds within the transaction. So sending 1 Ether to an account, it being sent back via smart contract, and sending it again would only result in a 1 Ether difference in the receiving account’s balance over the lifetime of the transaction.

If the proposer provided an address which was a smart contract that sent all funds to another address on receipt then this would result in the value of the block being under-reported, however this seems to be a limited use option and if a node operator attempted to do this it would be picked up pretty quickly and flagged. So I don’t see this as a reason to move to a significantly more complicated system of accounting.

(If it is really that much of an issue then you could trace the entire transaction to account for situations where funds move in and out of an account in the same transaction, but that would be relatively expensive and quite likely to be more open to abuse itself.)

9

Agreed with @jgm - imo, gross fee increase into fee-recipient directly or aggregated by tx both seem fine (and the former is probably easier to calculate). Trolling by setting fee-recipient as a looping contract just seems strictly inside the pareto curve to me - it’s obviously dumb if you’re trying to maximize from this block (your loop uses more gas so fewer txns can fit, relay has less room to optimize since sims take longer, relay pays you less by the increased basefee), and if you’re trying to grief then there are much easier ways to grief (eg, just don’t propose a block).

the only setup where this seems maybe incentive-compatible is in some enshrined pbs scenario where maybe you selectively grief relays that you don’t have an under-the-table deal with or something.

1 Like
10

arriving a bit late here, but totally see the merits of block level scoring. looking from the outside in, we intuitively came to the same approach a couple weeks back.

if the tradeoffs are not too costly to bear, “intuitive” as an attribute is def desirable, as it makes the score easier to replicate (e.g. when testing for correctness) for a larger number of parties—esp as this bleeds over to monitoring and reputation building.

4 Likes
11

Block level scoring does look good - nice and simple. One thing I’m a bit confused about (and perhaps it’s a mistaken cross-reference) - this discussion is tagged from this GitHub issue (Identifier for builder to proposer transaction · Issue #220 · flashbots/mev-boost · GitHub) which is seeking an identifier for the txn carrying the fees to the proposer’s fee recipient. How does the Block Level scoring address this?

12

here’s another for the bucket: https://twitter.com/siegerhino2/status/1573584907049246720?s=21&t=MHhODbE8lizLVBqpf6841Q

again thinking from the angle of relay reputation, should some variable that filters for the # of transactions included in the block be present in a scoring mechanism?

interested to hear people’s views here!

13

One benefit of plain block-level scoring is that the beacon-node could use eth_getBalance to easily double-check whether the received value equals the claimed value.

14

A downside would be that if there’s a transaction from the proposer’s fee recipient the block should literally be scored as negative eth.
Validation through the balance to me sounds like a patch that doesn’t actually address the problem, and works but for all the wrong reasons.

15

A problem with balance-diff block-level scoring is that a proposer can make a specific builder win the bids for their slot by providing a transaction that increases the feeRecipient balance (either directly or indirectly through smart contracts).

For reference: these are the current proposer payment mechanisms:

  • Last tx in block (tx output scoring): Flashbots, bloXroute, Eden
  • Setting proposer feeRecipient as block feeRecipient (balance-diff): Blocknative, Manifold
1 Like
16

After giving this much thought, we propose to standardize on payment transactions for block scoring, in particular in combination with merkle-proof of the payment transaction as part of the bid.

  • Payment transactions represent the actual block MEV value for the proposer, accurately reflecting the “bid value” a proposer uses to choose between multiple bids.
  • Currently 5 of 7 relays already use payment transactions. Their bids would loose out against bids with inflated value by a regular transaction to the proposer feeRecipient, even if the actual additional proposer value is higher.
  • mev-boost can easily verify the merkle-proof for inclusion without additional complexity, whereas verifying proof of balance difference would need mev-boost to additionally connect to an EL node.

Lastly it’s worth mentioning that there’s ongoing work to enable profit-switching in the beacon node (see also [RFC] Engine API: Add block value in response to engine_getPayload · Issue #307 · ethereum/execution-apis · GitHub).

5 Likes
17

I support this. We need a clear direction for next moves, and to me this a very good solution to explore.

I think this solves lido’s concern. I will reach out to get their confirmation.

Bloxroute’s concern was that this would require extra gas when the builder takes no payment. In this case, the proposer could be paid as fee recipient but we are forcing an extra transaction. I will also reach out for them to confirm this. I think the extra gas is a good price to pay for the simplicity and quickly enabling payment proofs and the local block comparison. We might find smarter ways to mitigate this case, too.

Thanks for the guidance @metachris!

18

I am unconvinced that the reasons provided are sufficient to claim payment transactions are better.

What do you mean by “actual block MEV value”? Surely any payment to the fee recipient, whether initiated by the builder or not, counts as MEV.

What is an “inflated value” and “actual additional proposer value”? Do you have examples where block level scoring behaves unexpectedly?

Why do you need to connect to an EL node for balance proofs? So long as you have the parent block header and the current block header, you can form a Merkle proof of the balance on the state trie which will be more accurate than attempting to prove a transaction against the transaction trie (see input and output scoring considerations).

19

Yeah, good point, what I meant was additional value delivered to the proposer by the builder.

It also relates to what we understand the bid value to be.

As an example:

  • Assume a block with searcher transactions that pay +10 eth to coinbase, and the builder transfers all of that to the proposer in the payment transaction
  • There’s also an transaction from the proposer feeRecipient transferring 11 eth out
  • Is the bid value now -1 eth? We argue the bid value should still be 10 eth

If builder A comes up with a value of 2 Ether and builder B comes up with a value of 2.5 Ether but includes a transaction that transfers 1 Ether to the proposer fee recipient this is a distortion of the value to the proposer.

(Also, having a reliable source of such transactions (i.e. pool payments) would allow to consistently outbid other builders/relays even if there’s the same actual proposer payment by the builder.)

Where would mev-boost get the parent header from?

  • Are you suggesting it’s supplied from the relay as part of the bid? can that be verified/trusted?
  • Ideally could you post a full example or spec of a bid with a verifiable balance proof?

Lastly, what’s your main argument for balance diff over payment transaction value?

20

Related discussion about details of payment proofs: Add bid proofs to the builder bid response by g11tech · Pull Request #51 · ethereum/builder-specs · GitHub