MEV Boost is used by all >85% of Node Operators, but at the moment there is little transparency to the reliability and performance of Relays. If a Relay were to stop providing bids, Node Operators and Builders would be unaware and unable to take corrective actions. For example, on November 10th, the Flashbots Mainet Relay experienced a DoS attack which resulted in a drop in MEV-Boost blocks between 12:00 - 16:00 UTC. Furthermore, existing dashboards are incomplete due to limited datasources, lack of real time data, and insufficient metrics. After connecting with Chris Hager from Flashbots, regarding Metrika’s Ethereum Data Challenge Blog, we discussed potential solutions to the Relay transparency problem that Flashbots had previously outlined. Additionally, Relays are a trusted party where builders must trust that their blocks are not shared with validators or other builders and validators must trust that Relays are verifying builders’ claims about potential blocks. At Metrika, we believe in transparency and accessibility therefore we are creating a MEV Relay Dashboard to help the community have access to basic data regarding the performance of Relays.
What
We are building (and releasing soon!) a free public MEV Relay Dashboard that will provide charts and visualizations of several metrics listed below grouped by Relays. The data for these dashboards and alerts will be collected via a modified version of open source the Relay Monitors.
The dashboard will display the following metrics:
Total bids - Ping each Relay for a bid in each Slot
Bid faults - The number of faulty bids submitted by a Relay. Currently we are tracking malformed and consensus invalid bids.
Relay latency - the latency of each Relay is dependent on the location of a Validator or Relay Monitor. We provide the average latency across several regions.
Time series of MEV blocks - The percentage of block on the chain that are MEV-Boost blocks vs naively built blocks
Users will be able to view the metrics by these 8 Relays (and we will add any additional Relays that come online):
I think insights into the builder ecosystem are even more interesting, rather than too much focus on the relays themselves. As long as they behave correctly, they are just dumb pipes.
But builders are the interesting parties…centralization/decentralisation, profit, loss, behavior, bidding strategies, … There’s a lot to explore!
In particular if the stats are not based on the anonymity of the pubkeys (without labels and attaching to organisations, they don’t give any insight who operates them). I feel using the extra_data is a helper here, and gives the builders a lot more tangibility and relatability.
The interesting part is who is running them, which parties have market share, what’s their strategies, profit/loss, etc.
Take also a look at relayscan.io for some reference.
And here’s a screenshot from a database tracking builder profit/loss, for inspiration (these are stats for the last 24h):
Thanks for the feedback! Yes, we agree that Builders are an interesting segment of the supply chain and we plan on adding Builder insight to our dashboard in the near future. Also, like you said, the Builder entity is key to making the information digestible for users. We decided to start with Relays monitoring for a couple reasons:
Relays are trusted parties that act as custodians of the entire system. We think it is important to ensure that Relays are performing their duties.
Although Relays are just pipes for Builders, Validators directly interact with Relays through their decision of which Relays to connect to
It could be useful for the community to have an external monitor as we have no conflict of interest. We don’t run Searchers, Builders, Relays, or Validators.
Lastly, here is our beta dashboard! This is just the start, we’ll be adding more soon.
On the other hand, missed bids cause no harm and are totally acceptable. Might just mean a relay is upgrading. It doesn’t seem a good reason to raise any alarms.
Yes, you are correct that Missed Bids don’t cause any harm and might not be a cause for alarm. Our thinking is that a high percentage of Missed Bids could be a leading indicator that a Relay is having issues. A Relay having issues may not be actionable information as Validators could be registered to multiple Relays and in the worst case the Validator builds a naive block. But given the trust placed in Relays, Validators should have data to evaluate Relay performance and make decisions. Based on the feedback from Fred, we looked at the % of missed bids over total registrations for the last week we can see different levels of performance for each Relay. We’ll be adding this table to the dashboard soon!
Registrations are Slots where the Proposer expects the Relay to provide a bid if the Relay received any blocks from its Builders.
Yes, Missed Bids do not necessarily imply there is an issue with the Relay. As you mentioned, we would expect each Relay to miss a % of bids for planned maintenance etc. It is interesting that some Relays have 10x more Missed Bids per Registrations than the best performing Relays. That difference could be explained by a couple possible reasons (not an exhaustive list):
Planned maintenance
The Relay did not receive a block from their Builders
The relay is having some issues
We want to highlight this difference between Relays as we expected % of Missed Bids per Registrations to be low and similar across Relays
