Trusting trust, and how to fight back

·10 min read

Reflections on Trusting Trust

In 1984 Ken Thompson accepted the Turing Award and used the lecture to explain, in about three pages, why you can't trust software. The paper is Reflections on Trusting Trust, and if you haven't already done so, read it. It's excellent and nasty all at the same time.

It basically boils down to the conclusion that you can't trust code that you did not totally create yourself.

This post explains the attack proposed by Thompson, and looks at some modern tools that (try to) fight back.

The attack

The attack targets a compiler in 3 steps:

Step one: a program that prints itself. You can write a program whose output is its own source code. This is called a quine. They have some interesting properties but importantly for this, a program can carry a copy of itself around and reproduce it on command. Hold that thought.

Step two: teach the compiler a backdoor. Say you want your compiler to miscompile one specific program. The C compiler has a function that compiles a line of source, roughly:

compile(line) {
  ...
}

You add a check to it:

compile(line) {
  if (matches(line, "the login program")) {
    emit("login, but also accept the password 'backd00r1'");
    return;
  }
  ...
}

Now whenever this compiler builds the login program, the binary it produces has a second, secret password baked in. The source of login is clean. Anyone can read it. The backdoor lives only in the compiler, and only appears in the binary the compiler emits.

This is already bad, but it's detectable. The extra if is sitting right there in the compiler's own source. Anyone reading the compiler would find it.

Step three: hide the backdoor from the compiler's source too. This is the beautiful but nasty bit. You add a second check:

compile(line) {
  if (matches(line, "the login program")) {
    emit("login, plus the 'backd00r1' backdoor");
    return;
  }
  if (matches(line, "the compiler itself")) {
    emit("the compiler, plus BOTH of these if-statements");
    return;
  }
  ...
}

The second check fires when the compiler is compiling itself. When it does, it re-inserts both backdoors into the new compiler binary. This is where the quine from step one becomes important: the compiler carries a copy of the malicious code and injects it into its own offspring.

Now do the following. Compile this poisoned source once to get a poisoned compiler binary. Then delete the two if-statements from the source code. Recompile the compiler using the poisoned binary. The new compiler comes out poisoned anyway, because the running binary put the backdoors back. From this point on:

  • The compiler source is clean. You can read every line. There is nothing there.
  • The login source is clean.
  • Every login binary has a backdoor.
  • Every compiler binary you build has the same self-propagating trap.

The bug lives only in the binary. You could audit every line of source on the machine and find nothing.

Reading source code won't save you

Even worse, the compiler is only the example. Thompson's point is that this extends down through every layer of abstraction we use.

You trust your program because you read the source. But you didn't run the source, you ran the binary the compiler produced. So really you trust the compiler. The compiler is itself a binary, produced by an earlier compiler. So you trust that one. Keep going down: the compiler, the assembler, the linker, the operating system that loads it, the CPU microcode that executes it, the silicon underneath. Thompson drops this cheerful bomb on us:

I could have picked on any program-handling program such as an assembler, a loader, or even hardware microcode. As the level of program gets lower, these bugs will be harder and harder to detect. A well-installed microcode bug will be almost impossible to detect.

Every layer is vouched for by the layer beneath it, and the bottom layer vouches for itself. Reading the source checks the very top and nothing underneath it.

This can feel a little academic. However, it is a very real attack vector. The xz-utils backdoor in 2024 was a multi-year effort to slip a backdoor into a compression library that ships with nearly every Linux box, where the payload hid in build artifacts (like binaries) rather than the source you can read on GitHub. The git history looked fine but the thing that got built and shipped was not. The reality is that most of us now run code we could never read, built by machines we don't control.

So, is it all doom and gloom? Is there a solution? Short answer is, kind of.

Some pragmatic defenses

There is no single thing we can do to mitigate this attack but there are some straightforward things we can do that help a lot:

Reproducible builds. Most software reaches you as a binary someone else compiled, and you take it on faith that it came from the source you can read. A reproducible build is one where the same source always produces the same bytes, on anyone's machine. So now you can independently verify by rebuilding from source yourself and confirming you get the identical binary. Alone it doesn't beat Thompson's attack above (without reading the next line, think about why...), because a poisoned compiler is deterministic too and emits the same poisoned bytes every time, but it sets the stage for the next defense, which relies on the fact that the same source always builds to the same bytes. The step from source to binary is no longer something you trust one builder about and becomes something anyone can verify.

Diverse double-compiling. This is the real answer to Thompson's compiler trojan, worked out by David A. Wheeler. The cool part is you never read the suspect compiler binary at all. Call the compiler you don't trust A, its source sA, and grab any second, unrelated compiler B. Then:

X      = A(sA)        # A compiling its own source (the "official" build)
stage1 = B(sA)        # rebuild A's source using the unrelated compiler B
stage2 = stage1(sA)   # use that rebuild to compile A's source once more

compare stage2 against X

If A is honest, stage2 and X come out byte-for-byte identical. If A hides a Thompson backdoor, they differ, and you can detect this.

The reason this works: if A is honest, A and stage1 are the same compiler but built two different ways. Both are just the source sA turned into a binary, one by A itself and one by B. The same program does the same thing, so compiling sA with either one produces the same bytes and thus stage2 and X will match.

However, a poisoned A breaks this. When A compiles, it adds the backdoor back in, something sA never asked for. stage1 doesn't, because B builds it honestly from sA. So A slips the backdoor into X while stage1 leaves stage2 clean, so the two no longer match, which makes the backdoor detectable. The one gotcha is if B hides the exact same backdoor, i.e. someone has deliberately backdoored B to inject the identical trojan. That's theoretically possible but unrealistic, especially if we add more compilers to our list. (This all rests on the reproducible-builds guarantee above: the same source always builds to the same bytes.)

Bootstrappable builds. Even so, there's still a floor. Every compiler was built by an earlier compiler, which was built by an earlier one, back to some original binary nobody ever audited. You can't get rid of that starting binary, you always need something to compile the first thing with, but you can make it tiny. Bootstrappable builds start from a seed small enough for a person to read, then build the whole toolchain up from source on top of it one step at a time. GNU Guix's full-source bootstrap begins with a 357-byte program written in hex, small enough to check by hand, and from there builds the entire toolchain up from source (357-byte hex seed → tiny assembler → GNU Mes → TinyCC → GCC → full toolchain). The binary you have to take on trust drops from a full compiler of unknown origin to a few hundred bytes you can read.

Put the three together and the software stack is covered pretty well: the source is readable, the source-to-binary step is reproducible and checkable by anyone, the compiler is proven honest without trusting its binary, and the seed underneath is small enough to audit by eye. The catch is that this all leans on diversity. Diverse double-compiling needs a second, unrelated compiler to check against, and reproducible builds only mean something if independent people actually rebuild and compare, not just the original author. If the whole world builds everything with one compiler lineage, a single poisoning will win everywhere and there's nothing to cross-check against. So really, monoculture is the risk here. Keeping independent implementations of the important tools alive is part of the defense.

One last gap: the machine itself. So now we have done a good job of covering the code, but all of it assumes one more important thing: that you're running the build and the checks on hardware you trust. This is very often not the case, these days the code usually runs on someone else's server, where the operator could swap your binary or read its memory while it runs. Strictly speaking this is a different attack compared to what Thompson raises but it's a real issue in 2026 so let's see if we can close that gap also with a different tool: a trusted execution environment (TEE), or enclave, plus remote attestation. The way it works is the CPU signs a hash of exactly what it loaded, so a remote user can check the running binary really is the one they audited, and the operator can't tamper with it without us knowing.

By itself that only proves which binary is running, not that the binary is any good. An enclave will attest a Thompson-poisoned binary just as happily as a clean one. So we pair it with the three steps from above which are what make the binary trustworthy in the first place, and reproducible builds calculate the exact hash to expect, so when the enclave proves that same hash is what's really running, we've carried all that trust onto a server we don't own. Together they close the loop: an honestly-built binary we can tie back to source, and proof it's the one actually executing, even on someone else's machine.

The one catch is that leaning on an enclave moves our root of trust onto the CPU vendor's silicon (the thing that hashes the code and signs off on it with a key only it holds), so we haven't escaped trust altogether, just shrunk it down to something smaller. But for the one problem the three steps can't solve, running on a machine we don't control, it's one of the best tools we've got.

Where this bottoms out

We never get to zero trust. Thompson was certainly right about that. What's changed in the forty years since is how much we have to trust, and how much of it we can actually verify. Instead of blindly trusting every compiler, build box and binary in the chain, we've now got source anyone can read, builds anyone can reproduce, a compiler checked against a rival so it can't lie, and a seed we can read ourselves. We're still trusting plenty on top of that, the CPU running the enclave, the vendor who made it, and that at least one of our compilers is honest. We haven't removed trust, but we have shrunk it.

Thompson said you can't trust code you didn't create yourself. He's still right. But we are in a strictly better position today than we've ever been before.

Read the original lecture if you haven't.