Related:
TEE web services
There’s a curious problem in the BuilderNet which we did not quite solve to our liking yet. How do you request a TEE web service exactly?
As most web services, a TEE web service is served by some host on some domain over TLS. The difference is that we can request the TEE web service to provide us with a proof that it’s running the code we expect on an up-to-date hardware — through remote attestation.
Attested TLS and remote attestations
How we handle remote-attested TLS connections is through a TLS-terminating reverse proxy, which both the client and the server run, and which performs remote attestation to ensure TLS is terminated by the TEEs and not a middleman. Oversimplified remote attestation protocol:
- Challenge nonce is sent to the TEE
- Challenged TEE will produce an attestation report with that nonce
- Counterparty will verify that the attestation report is correct:
3a. TEE is up-to-date
3b. TEE is running the expected code
As the attestation report in (2) contains the TLS pubkey, at the end of this handshake we have established a TLS connection that terminates inside the TEE that generated the report (guaranteed by DH handshake) and that runs the code we expect.
Note#1: either or both the client and the server can be running in TEEs, but that’s less important. From now on I’ll assume it’s the server that’s in a TEE.
Note#2: In this post I’ll ignore (3a.) and instead focus on (3b.). Verifying TEE is up to date is far from trivial, but is more related to specific providers than to applications.
Requesting a TEE web service
Assume a common scenario, in which I (user) want to send a http GET request to a TEE web service. What do I do?
First, I need to establish the expected measurement. I should check with the app’s developers which version is currently running, rebuild the necessary code (and image) and from it extract the expected measurement. The format our reverse proxy expects looks as follows:
[
{
"measurement_id": "azure-tdx-example-01",
"attestation_type": "azure-tdx",
"measurements": {
"4": {
"expected": "ea92ff762767eae6316794f1641c485d4846bc2b9df2eab6ba7f630ce6f4d66f"
},
"9": {
"expected": "c9f429296634072d1063a03fb287bed0b2d177b0a504755ad9194cffd90b2489"
},
"11": {
"expected": "efa43e0beff151b0f251c4abf48152382b1452b4414dbd737b4127de05ca31f7"
}
}
}
]
I should then run my client-side attested TLS proxy configured to the TEE’s host, and finally through the proxy send my http GET. That’s a lot of work, and I’m assuming the currently running version has been audited and is safe — which doesn’t need to be the case!
How do we do better?
Measurements and hosts go together
There’s no reason for the TEE host and the measurement to be separate. Something as simple as using https://<expected measurements specifier>@host.domain. Gzipped and base64 encoded measurement passed as userinfo would look as follows:
https://H4sIAAAAAAAAA82QTWoDMQyF9zlFmHUCsmxZnl6llCBZMhSaNjRTSCm5e53ZJKGQdFOoFsbo93vvcbHs8TW/pxi2LvuPd9/667R5tuFhOViV3Xqyw9oPst29+BpwWJ0HZJp8P8n0/Pa6mT53fjly2XexeN97zifnKvxIzWk/7LxOPnOQUYYCrqO3Uh1UpWW1amLuCtm5koIkC15DVi+5SCEZKaqoQyWnZBBqcSAQSs1DFeASGSmC4XB1/ri6Bgz3AROG7CNS4zG35FiwNulYaKkyS4HOPxaWJjgGIsmhWY1RTLsqUbAUAooaYSzs5NhboGtrWoSw8G1A/A0gd9VNeYQRu0cJAJQ4aE5eNUXsZJG4A3UpzcJMnTnkYmzcuHnBrq0brHLawN3YSLWl2Mgr0W3AeB8Q/jhuA6Z/B7i4/h0XT4tv+8/CCrEDAAA=@host.domain
It’s a bit long — but you can copy-paste it, and your proxy can now simply parse it and forward the requests to expected hosts. The behaviour is more or less the same, but it could be a bit more convenient in some use cases.
They who control the versions control the measurements
Here I’ll be expanding on Portrait of a TEE’s notion of a TEE application being simply its governance mechanism (basically upgrades).
Which version should be considered safe is the biggest and most important issue that application’s governance has to resolve. For centralized or permissioned applications this is usually done by the development team on github — by no means safe, but with the releases being approved by multisigs and such we have grown accustomed to this.
Decentralized or permissionless applications are usually governed by smart contracts — a more complicated version of a multisig that will usually (hopefully) also involve users and auditors in some way.
If we trust the governance to versions — there’s really no reason not to trust them with managing expected measurements. Malicious governance that controls which version is considered safe will introduce backdoors regardless of whether it also controls expected measurements.
As long as users are not forced to only trust governance — as long as they can still reproduce the artifacts from scratch — I could see this being safer for most users, who will not be reproducing the artifacts themselves.
This would let us address TEE applications rather than specific versions of artifacts, and the URLs would look as follows:
https://aHR0cHM6Ly9idWlsZGVybmV0LmZsYXNoYm90cy5uZXQvbWVhc3VyZW1lbnRzCg==@buildernetip.domain
https://0xde374479CEe0792F9c3fD389E5c28B8574b0a7DD@host.domain
In case of multisigs we could also of course simply verify the signatures, adding the expected pubkey to userinfo of the url.
Upgrades to applications are now transparent — and still as secure as the governance of the application. Which is extremely useful in some circumstances (in some others you maybe wouldn’t want that). The attack surface area is arguably bigger than if the user follows all the steps to reproduce an image — but most users won’t. This will be safer for users that don’t reproduce their measurements, as the measurements will be retrieved directly from the app’s governance mechanism rather than from individual developers, or the internet, or telegram groups.
Your TLS terminating proxy would parse those, and request expected measurements from the source you specified. This is more or less what is currently done in the permissioned BuilderNet, with users fetching expected measurements from the application’s developers service. Your attested TLS proxy can do it now too.
You know what’s better than a TLS proxy? TLS with no proxies.
All of the above still needs either a proxy or a library that modifies TLS handshake to remotely attest the handshake. You can’t do that in a browser!
If the user doesn’t remotely attest the TEE during TLS handshake — we could still perform a post-establishment handshake separately. With non-proxied TLS being such a UX improvement I could see all of TEE web services exposing a handshake endpoint in their API — however, it’s far from trivial to make sure this is safe from man-in-the-middle attacks.
The user not performing a remote attestation is inherently less secure — as users will be only indirectly verifying the attestation. But it could be secure enough — and still worth considering, as it would be the simplest and the preferred way to interact with any web service. It would also be compatible with browsers, which is really important!
For how non-proxied TLS for TEEs could work, see the excellent Securing Domain Certificates: Ensuring Exclusive Control by TEE by @shelven and GitHub - Dstack-TEE/dstack from Phala.
Basic idea is to control a domain from within a TEE — and sign subdomain certificate requests according to requestor TEE’s measurements! This falls somewhere between the ideas presented above, as it works on specific versions’ measurements rather than applications — but is as convenient as a centralized service. This approach could also benefit from moving to IDs of applications’ governance from IDs of specific versions of container manifests.
Takeaways
- Addressing TEE web services is far from trivial. It’s also inevitable and the biggest challenge your users will face.
- Please don’t post expected measurements in random places on the internet. Your users will at some point request the wrong version of your app — hopefully just outdated rather than malicious.
- An application is its governance, and governance over upgrades, safe versions and releases is the problem at the core of TEE applications. Expected measurements of a TEE service is downstream of governance over versions and upgrades. And this should be reflected in how you address your TEEs.
- I’d suggest to address your TEE apps in a way that matches the governance they follow. Permissioned apps should expose their releases, and therefore expected measurements, in a permissioned way — on multisig contracts, signed GitHub releases, regular old https endpoints. For permissionless apps — both the governance and addressing should be contract-based.
- BuilderNet should publish releases (and therefore expected measurements) in a multisig contract for transparency even if it’s permissioned!