The Man Who Didn't Belong

Feynman was a strange choice for a presidential commission. A theoretical physicist, Nobel Prize winner in 1965 for quantum electrodynamics, Manhattan Project as a young man, bongo drums, safe-cracking at Los Alamos for fun. He was famous for asking dumb-sounding questions until the smart people in the room realized they didn't have the answer either.

That's the Feynman the public knew. There's a second one that matters more here. He was a showman. He understood live audiences. He knew that a fact, dropped at the right moment, lands harder than the same fact buried in a 200-page report nobody reads. And he was politically savvy.

The day of the demonstration, the man next to him was General Donald Kutyna. When Feynman first reached for his microphone, Kutyna whispered for him to wait. A few minutes later, Feynman reached again. Not yet. Wait until the official is on this specific slide. Feynman waited. When the slide came up, he pressed the button. [2]

That moment didn't happen because a brilliant scientist couldn't help himself. It happened because a brilliant scientist and a smart general timed it together. Without that timing, the truth would have died in a footnote.

The Note

There's a piece of this that stayed buried for almost thirty years.

Sally Ride was on the commission too. First American woman in space, and an astronaut who knew the shuttle from the inside. Early in the investigation, someone at NASA slipped her a document: two columns, temperature in one, O-ring resiliency in the other. The colder it got, the stiffer the rubber. The whole case, on a single page.

She couldn't use it. Not openly. If the first American woman in space stood up in a televised hearing and named the O-rings, she'd have ended her own career and burned the engineers who handed it to her. So she passed it sideways.

Walking next to Kutyna one day, eyes straight ahead, she opened her notebook and slid him the page with her left hand. Never said a word. She trusted him to carry it the rest of the way without naming her. [3]

Kutyna had the same problem. He couldn't reveal the source either. So he staged it. He had Feynman over for dinner, walked him out to the garage to admire a 1973 Opel that Feynman couldn't have cared less about, and pointed at the carburetor he'd been cleaning. These have O-rings, he said. When it gets cold, they leak. Think that has anything to do with our situation? Feynman didn't say a word. The next Tuesday, he asked for ice water. [4]

Three people knew the answer. None of them could say it directly. An astronaut, a general, and the engineers who started the chain. The truth reached the public, but only after it had been routed through a dinner conversation about a carburetor.

This is a story about an organization where speaking the truth could cost you the career you'd spent a lifetime building.

The Bureaucratic Sandwich

Challenger broke apart 73 seconds after launch on January 28th, 1986. Seven people died, including a high school teacher chosen to be the first ordinary citizen in space. The country watched live.

Reagan appointed a commission of about a dozen members, chaired by former Secretary of State William Rogers. Mostly senior executives, retired generals, former officials.

Feynman did something none of the others did. He insisted on talking directly to the engineers who built the shuttle. Not the managers. Not the program leads. The engineers.

Early on, he wanted to know how the rubber O-rings in the booster joints behaved in the cold. The launch had happened on a freezing morning. Temperature was a key factor. He sent the question to NASA.

The answer came back as what he called a sandwich of memos.

Top layer: "Professor Feynman wants to know about the O-rings."

Every layer below was a subordinate passing the question down.

At the bottom, finally, a page of numbers. They answered the wrong question. He'd asked how the rubber responded in milliseconds, the time scale of a launch. The numbers described how it crept back over hours. It was useless information. [5]

That's when Feynman stopped asking through proper channels.

The next morning at 8 a.m. it was snowing in Washington. He stood outside a hardware store in a suit, waiting for it to open. He bought a few tools and the smallest C-clamp he could find. He took them to the hearing, pulled a sample of the booster joint rubber, and waited for ice water.

The Fantastic Figures

Here's where the story gets relevant for leaders.

While he investigated the O-rings, Feynman started asking a different question. What did NASA think the odds of a catastrophic shuttle failure actually were?

The gap was massive. Working engineers put it at roughly 1 in 200. Management claimed 1 in 100,000. [6]

The clearest example came from the main engines. Feynman asked the engineers who built them to write down their estimates. The most optimistic was 1 in 300. Then he asked their manager. The manager refused to give a number and wrote a paragraph instead, about "engineering judgment, past experience, and quality control."

That phrase, "engineering judgment," amounts to hand waving. It sounds technical. It sounds responsible. What it means is, "We don't have to give you a number. Trust us."

Feynman pushed harder. The manager eventually said the odds of mission success were "100 percent minus epsilon." Feynman asked what epsilon was.

The answer was: 10 to the negative 5. One in 100,000.

That means you could fly the shuttle once a day for nearly 300 years between accidents. No serious person who had actually worked on the engines would commit to that estimate. It's the number that comes out the backside of a system unwilling to grapple with reality.

That's the pattern. Engineers screaming, in his words, "HELP, this is a RED ALERT." Management quietly loosening criteria, accepting more errors, keeping the schedule. Until something broke.

History proved the engineers right within an order of magnitude. Management was off by three. There were 135 shuttle missions and two catastrophic losses. Challenger in 1986. Columbia in 2003.

Appendix F

Feynman wrote up what he saw as a personal appendix to the Rogers Commission report. It's a short, fifteen-minute read. It is one of the most straightforward pieces of writing about engineering culture that I've read. I highly recommend it.

He almost didn't get it included. He had to fight for it. He threatened to have his name removed from the commission report if his findings weren't included.

He opens by stating plainly that the failure estimates ranged from 1 in 100 to 1 in 100,000, the high numbers from engineers, the low ones from management. Then he asks the question that frames the whole document. What is the cause of management's fantastic faith in the machinery?

The rest of Appendix F answers it. He describes "an almost incredible lack of communication" between managers and their working engineers. He describes how safety criteria get altered, with arguments that sound logical, until a system designed to be very safe is flying in a relatively unsafe condition.

And then, at the very end, the line:

For a successful technology, reality must take precedence over public relations, for Nature cannot be fooled. [7]

That attitude, the commitment to launch at all costs, is all too prevalent in product security right now.

Seventeen Years Later

Feynman published that warning in 1986. NASA read it. Then, slowly, the culture he described grew back.

In 2003, Columbia broke apart on reentry and killed all seven aboard. The Columbia Accident Investigation Board went looking for the cause and found something familiar:

By the eve of the Columbia accident, institutional practices that were in effect at the time of the Challenger accident – such as inadequate concern over deviations from expected performance, a silent safety program, and schedule pressure – had returned to NASA. [8]

The exact practices. Inadequate concern over deviations. A silent safety program. Schedule pressure. The board was describing the Challenger failure, back from the grave.

That's the part that should scare you. NASA didn't miss the lesson. They had it in writing, from a Nobel laureate, in their own report. They fixed the practices, and within a generation the practices came back, because the thing underneath was never fixed: an organization that doesn't want to hear bad news.

A control you fix once does not stay fixed. Culture reverts to whatever the incentives reward.

The Same Pattern, Different Industry

Feynman documented three things at NASA. Engineers knew about the problem. Management didn't want to hear about it, especially if hearing it meant delaying the schedule. And the bar for "safe" got quietly lowered over time.

That's the same pattern I've watched from the inside in multiple industries. There's a documented public version of it too, in self-driving cars.

In 2019, Tencent's Keen Security Lab published research on Tesla Autopilot with three findings. They tricked the auto-wipers into activating with adversarial images on a display in front of the windshield. They tricked the lane recognition into following a fake lane with small stickers on the road, and the car drove into oncoming traffic. And they took over the steering with a wireless gamepad, even when Autopilot was off. [9]

Now read Tesla's responses, as recorded by Keen Lab, with Feynman's NASA in your head.

On the wipers: the test "is not a real-world situation," it used a display in front of the windshield, and the feature is in beta. On the lane recognition: the researchers "adjusted the physical environment" with tape, and a driver "can easily override Autopilot at any time." On the steering: the vulnerability was fixed by a "robust security update" in 2017, and Tesla had "never seen a single customer ever affected."

Three findings. Three deflections. Not a real-world situation. The driver can override. Already patched, no customer affected.

Read that last one again. "No customer ever affected."

Feynman had a line for that. Talking about NASA treating O-ring erosion on past flights as evidence of safety, he called it a kind of Russian roulette: the fact that the first shot got off safely is of little comfort for the next. "No customer ever affected" is the same logic. The chamber just hasn't come up loaded yet.

That's the modern "engineering judgment." A verbal substitute for engaging with the finding.

I'm not claiming Tesla didn't fix the issues. They probably did. That's not the point. The point is the tone, the response of an organization that clearly doesn't want to hear about the problem.

NASA said 1 in 100,000 because they didn't want to hear 1 in 300. Tesla said the driver can override because they didn't want to engage with researchers who hijacked the car. Both responses are technically defensible if you squint. Both are defensible only because the system hasn't yet failed publicly enough to make them indefensible.

The Design Question

That's the pattern your security teams are dealing with right now. They surface findings. They get push back with some version of "not a real-world situation." They're burning out trying to be heard.

In recent videos I've offered two questions to take into design reviews. Does this control still work when no human is watching? Does it still work when the model has been compromised?

Here's one more.

Does this control survive contact with the people who don't want to hear about problems?

If your finding only lives in places management controls, they can erase it with a single statement. That's what Tesla did. That's what NASA did. The control that survives is the one in the architecture, in the test plan, in the regulator's filing, in the public record.

Sally Ride understood this in her gut. She couldn't win the argument inside NASA, so she didn't try. She moved the finding to the one place NASA management couldn't edit: a glass of ice water on live television.

Feynman knew that. It's why he insisted on talking to the engineers. It's why he wrote Appendix F instead of trusting the main report to carry the message. It's why he timed the ice water for live television.

He was building controls that didn't depend on management wanting to hear them.