The Written Prediction Audit
Record dated forecasts across asset classes, then grade yourself on a reminder to find out how bad your instincts actually are.
- Difficulty
- Starter
- Time to result
- ~months to results
- Steps
- 5
- Confidence
- 85%
This is Tim Ferriss's addition to the async allocation discussion, and it is a calibration instrument rather than an investing method. The input is a fixed annual date, which he says could be January 1 or January 5 and genuinely does not matter as long as it is fixed. On that date you sit down, pick a time frame such as six months out, and grab a bundle of potential asset classes, stocks, or indices, his own example being Bitcoin, a barrel of oil, and the S&P 500. You make explicit written predictions for each. You then set a reminder to look at them again three or six months later. The mechanism is the gap: over that gap you are influenced subconsciously by chatter on Twitter and trends in the news into believing you know where things are going, and the written record is the only thing that survives that influence unchanged. The output is a score, and the score is the point. Ferriss frames it as a way to check your cognitive biases, and it works because it converts a diffuse feeling of insight into a countable hit rate. It is deliberately cheap: one sitting a year, a handful of lines, one reminder.
Origin
Ferriss offered this on-air as an extension of Srinivasan's async decision rule, drawing on his own repeated observation that he and people around him consistently overestimate their predictive abilities and never find out because nobody keeps the receipts.
Core principles
- 01Humans systematically overestimate their predictive abilities, and the overestimate is invisible without a written record.
- 02A prediction is only auditable if it is dated, specific, and made before the outcome window opens.
- 03Chatter on social media and trends in the news influence your sense of certainty subconsciously, so confidence is a poor proxy for accuracy.
- 04Grading a batch of predictions across several asset classes at once separates skill from a single lucky call.
- 05The purpose of the audit is calibration, not performance; the output is a better prior about how much to trust yourself.
How to run it
- 1
Fix an annual prediction date
Choose one date a year, such as the first week of January, and put it in the calendar permanently. The specific date is irrelevant; the fixedness is what matters.
Pro tip Attach it to an existing annual ritual so it never gets skipped.
Watch out A floating date turns into no date, which is how this habit dies.
- 2
Declare the forecast horizon
State the window explicitly, for example six months from now. Every prediction in the batch shares the same horizon so they can be graded together.
Pro tip Six months is long enough to escape noise and short enough to close the feedback loop within a year.
Watch out Mixing horizons in one batch makes the eventual scoring incomparable.
- 3
Predict across a spread of asset classes
Write directional predictions for several different things at once, such as Bitcoin, a barrel of oil, and a broad index, so the batch measures a general ability rather than one call.
Pro tip Include at least one asset you feel uncertain about to expose the difference between confidence and accuracy.
Watch out Predicting only what you already hold turns the audit into a rationalisation exercise.
- 4
Set the review reminder immediately
Before closing the session, set reminders at the three and six month marks pointing back at the written predictions.
Pro tip Put the predictions themselves into the reminder body so no lookup is required.
Watch out Without the reminder you will only remember the predictions that came true.
- 5
Grade and recalibrate
On the review date score each prediction, count the hit rate, and explicitly adjust how much weight you give your own macro read in future decisions.
Pro tip Note which predictions you would have restated more confidently mid-window, since that gap is the bias you are hunting.
Watch out Retroactively reinterpreting a vague prediction as correct destroys the entire measurement.
In the wild
Ferriss's own worked example is to sit down once a year, pick six months as the window, and make written calls on Bitcoin, a barrel of oil, and the S&P 500. Six months later, having spent the intervening period absorbing Twitter chatter and news trends that quietly build a sense of knowing where things are going, he reopens the file and checks how successful he actually was.
→ The exercise produces an honest hit rate that either justifies acting on macro conviction or, more often, argues for discounting it.
Common mistakes
Making predictions without setting the review reminder
Unreviewed predictions are worse than none, because memory selectively retains the hits and the exercise ends up increasing confidence rather than calibrating it.
Writing predictions too vaguely to be graded
A forecast like things get choppy can be scored as correct in almost any outcome. Only directional, dated, specific claims produce a usable score.
Treating the score as a performance review rather than a calibration input
A low score is the useful outcome, since it tells you how much to discount your own instincts. Reacting to it with embarrassment leads people to abandon the audit exactly when it starts working.
Is it for you?
Best for
Anyone who makes allocation or strategy decisions on the strength of their own read of where things are heading.
Not ideal for
Purely passive investors who never act on a directional view and therefore have no forecasting habit to calibrate.
From the transcript
“humans in general and i succumb to this sometimes to over estimate their predictive abilities so what is very helpful is set a time it…”
“you grab a bunch of potential asset classes or stocks or indices so maybe it's bitcoin a barrel of oil s p 500 and make…”
“let's see how successful you were over the preceding six months so that's also a good way to check your cognitive biases”
From the episode
#606: Balaji S. Srinivasan — 5-10-Year Predictions, How to Start a New Country, Society-as-a-Service (SaaS), Bitcoin Maximalism, Memetic Warfare, How Prices Are Born, Moral Flippenings, The One Commandment, and The Power of Missionary over Mercenary