MEV Relay Monitor Dashboard

Why

MEV Boost is used by all >85% of Node Operators, but at the moment there is little transparency to the reliability and performance of Relays. If a Relay were to stop providing bids, Node Operators and Builders would be unaware and unable to take corrective actions. For example, on November 10th, the Flashbots Mainet Relay experienced a DoS attack which resulted in a drop in MEV-Boost blocks between 12:00 - 16:00 UTC. Furthermore, existing dashboards are incomplete due to limited datasources, lack of real time data, and insufficient metrics. After connecting with Chris Hager from Flashbots, regarding Metrika’s Ethereum Data Challenge Blog, we discussed potential solutions to the Relay transparency problem that Flashbots had previously outlined. Additionally, Relays are a trusted party where builders must trust that their blocks are not shared with validators or other builders and validators must trust that Relays are verifying builders’ claims about potential blocks. At Metrika, we believe in transparency and accessibility therefore we are creating a MEV Relay Dashboard to help the community have access to basic data regarding the performance of Relays.

What

We are building (and releasing soon!) a free public MEV Relay Dashboard that will provide charts and visualizations of several metrics listed below grouped by Relays. The data for these dashboards and alerts will be collected via a modified version of open source the Relay Monitors.

The dashboard will display the following metrics:

  • Total bids - Ping each Relay for a bid in each Slot
  • Bid faults - The number of faulty bids submitted by a Relay. Currently we are tracking malformed and consensus invalid bids.
  • Relay latency - the latency of each Relay is dependent on the location of a Validator or Relay Monitor. We provide the average latency across several regions.
  • Time series of MEV blocks - The percentage of block on the chain that are MEV-Boost blocks vs naively built blocks

Users will be able to view the metrics by these 8 Relays (and we will add any additional Relays that come online):

  1. Flashbots
  2. BloXroute Max Profit
  3. Eden
  4. Blocknative
  5. BloXroute Ethical
  6. BloXroute Regulated
  7. Manifold
  8. Relayooor

Future considerations

Some ideas we have for the future are adding more metrics and alerts:

Future metrics:

  • Normalized profitability of relays
  • Comparison of Actual vs Expected pay from Builders/Relays

Alerts that will be provided:

  • Consecutive Missed Bid - Alert if a Relay misses five consecutive bids
  • Bid Fault Alert - Alert if a Bid Fault occurs
  • Relay Latency - Alert if a bid request takes more than 0.5 second

Call to action

We’d like to invite the community to provide feedback on the metrics we are displaying. A couple open questions:

  • Are these the most important and useful metrics? What are we missing?
  • What is the best methodology to calculate these metrics?
  • What else might be useful for you in the future?
  • Want early access to a beta version? Get in touch here
5 Likes

can’t access to the touch link

1 Like

Hi try this link

Very nice!

I think insights into the builder ecosystem are even more interesting, rather than too much focus on the relays themselves. As long as they behave correctly, they are just dumb pipes.

But builders are the interesting parties…centralization/decentralisation, profit, loss, behavior, bidding strategies, … There’s a lot to explore!

In particular if the stats are not based on the anonymity of the pubkeys (without labels and attaching to organisations, they don’t give any insight who operates them). I feel using the extra_data is a helper here, and gives the builders a lot more tangibility and relatability.

The interesting part is who is running them, which parties have market share, what’s their strategies, profit/loss, etc.

Take also a look at relayscan.io for some reference.

And here’s a screenshot from a database tracking builder profit/loss, for inspiration (these are stats for the last 24h):

2 Likes

Thanks for the feedback! Yes, we agree that Builders are an interesting segment of the supply chain and we plan on adding Builder insight to our dashboard in the near future. Also, like you said, the Builder entity is key to making the information digestible for users. We decided to start with Relays monitoring for a couple reasons:

  • Relays are trusted parties that act as custodians of the entire system. We think it is important to ensure that Relays are performing their duties.
  • Although Relays are just pipes for Builders, Validators directly interact with Relays through their decision of which Relays to connect to
  • It could be useful for the community to have an external monitor as we have no conflict of interest. We don’t run Searchers, Builders, Relays, or Validators.

Lastly, here is our beta dashboard! This is just the start, we’ll be adding more soon.

2 Likes

The data of missed/failed bids per Relay is super cool to see. I haven’t seen data on this visualized anywhere else.

It would be interesting to see this in percentage format, i.e “How often does Relay X miss a bid?”

Very nice dashboard!

1 Like

Thanks for the feedback! We’ll work on adding the percentage of missed bids per relay

1 Like

On the other hand, missed bids cause no harm and are totally acceptable. Might just mean a relay is upgrading. It doesn’t seem a good reason to raise any alarms.

1 Like

Yes, you are correct that Missed Bids don’t cause any harm and might not be a cause for alarm. Our thinking is that a high percentage of Missed Bids could be a leading indicator that a Relay is having issues. A Relay having issues may not be actionable information as Validators could be registered to multiple Relays and in the worst case the Validator builds a naive block. But given the trust placed in Relays, Validators should have data to evaluate Relay performance and make decisions. Based on the feedback from Fred, we looked at the % of missed bids over total registrations for the last week we can see different levels of performance for each Relay. We’ll be adding this table to the dashboard soon!

I don’t think it’s a good indicator for that though. Could as well be planned maintenance, etc.

What’s “registrations” in this table?

Registrations are Slots where the Proposer expects the Relay to provide a bid if the Relay received any blocks from its Builders.

Yes, Missed Bids do not necessarily imply there is an issue with the Relay. As you mentioned, we would expect each Relay to miss a % of bids for planned maintenance etc. It is interesting that some Relays have 10x more Missed Bids per Registrations than the best performing Relays. That difference could be explained by a couple possible reasons (not an exhaustive list):

  1. Planned maintenance
  2. The Relay did not receive a block from their Builders
  3. The relay is having some issues

We want to highlight this difference between Relays as we expected % of Missed Bids per Registrations to be low and similar across Relays