The False Proxy Audit
Stop measuring the easy signal and measure the thing you actually want
- Difficulty
- Moderate
- Time to result
- ~weeks to results
- Steps
- 5
- Confidence
- 87%
A false proxy is easy to measure and ultimately useless. Typing speed is easier to measure than programming ability, so we measure typing speed. Worse, we measure whether the candidate looks like us: Godin cites a company whose last nine hires had rowed varsity crew at one of three colleges. The audit has two moves. First, replace proxies with direct evidence: Godin now only works with people he has worked with, meaning he pays them for a real project and that project is the interview, which incidentally widened his talent pool geographically and demographically. Second, insulate decision makers from information that corrupts their horizon: a fund promising five-year thinking should not have a Bloomberg ticker on the wall. Finally, require the reasoning to be spoken aloud so peers can judge the decision on its merits.
Origin
Extracted from The Tim Ferriss Show
Core principles
- 01We measure what is easy, not what matters
- 02Familiarity masquerades as merit
- 03The best proxy for doing the job is doing the job
- 04Information you cannot act on distorts the decisions you can make
- 05Every proxy should be defensible out loud
How to run it
- 1
Inventory the signals you actually use
For one recurring decision, list every signal that influences it, including the informal ones nobody writes down.
Watch out The most powerful proxies are usually the unstated ones.
- 2
Trace each signal to the outcome it stands for
For each, name the real outcome you care about and honestly assess the link. Typing speed to programming quality is the canonical broken link.
Pro tip Any signal that mainly detects similarity to you is a lazy shortcut.
- 3
Substitute a work sample
Replace the strongest false proxy with direct evidence: pay the person to do a scoped version of the actual work and treat that as the evaluation.
Pro tip You never even need to meet in person, which widens the pool enormously.
Watch out Being good at interviews is a skill you are not hiring for.
- 4
Insulate the decision from useless information
Remove data streams whose timescale contradicts the horizon you promised. A five-year mandate is undone by a live ticker on the wall.
Watch out Measuring the wrong thing in the wrong timeframe confuses everything downstream.
- 5
Make the rationale sayable
Require anyone making the call to state why, so peers can debate the decision rather than wait for the result.
Pro tip Show your cards, make your argument, make your assertion.
Watch out 'I just feel like it' is not a decision-making process.
In the wild
Godin admits he was arrogant about hiring: he could read certain interview signals faster than most people, and mistook that speed for accuracy. On reflection, what he wanted from a hire was for them to do the job, not to be good at being interviewed. So he switched to only working with people he had worked with, which means paying them for a real project and treating that project as the interview. No meeting is required.
→ A far more geographically and demographically diverse set of collaborators, evaluated on the actual work.
On the Carbon Almanac, every volunteer could do anything they wanted once, and if the community liked it they got to do more. Vivek, from India, wrote one article and it was terrible. Someone gave him feedback and the second was better. He was about to quit, got more feedback, and the third was so good he ended up writing seventeen of the articles. No credential or interview would have surfaced him.
→ One of the project's most prolific contributors, found by letting the work be the filter.
Common mistakes
Hiring people who look like you
Selecting for candidates who resemble your existing team, such as nine consecutive varsity crew rowers, is a lazy shortcut dressed up as a standard.
Keeping a ticker on a five-year plan
Feeding decision makers real-time data when they have promised long-horizon judgment guarantees they will optimise for the wrong timeframe.
Is it for you?
Best for
Anyone who hires, promotes, funds or evaluates work and suspects their criteria are tracking convenience rather than capability.
Not ideal for
Highly regulated selection processes where the criteria are externally mandated and cannot be changed.
From the transcript
“False proxies are easy to measure but ultimately not useful. How fast someone types is a false proxy for whether they're going to be a…”
“If I'm going to hire you, I'm going to give you a project and pay you to do it, and that's your interview.”
“Show your cards, make your argument, make your assertion, then your peers can talk to you about whether that's a good decision or not.”
From the episode
#792: Seth Godin on Playing the Right Game and Strategy as a Superpower