Calibration: does your ‘sure’ match how often you’re right?

Imagine your friend Arjun makes lots of little bets in school. Every time, he says how sure he is. "I am 90% sure it will rain today." "I am 90% sure our team will win." "I am 90% sure this answer is correct." Arjun sounds very confident every time. But when you actually count, you find that things Arjun said he was "90% sure" about only came true about half the time.

Something is wrong. Not that Arjun is stupid - he is often clever. The problem is that his sureness does not match reality. When he says 90%, the true chance is much lower. His confidence dial is broken.

Philip Tetlock, the researcher who studied who guesses the future well, cared deeply about this exact problem. He gave it a name: calibration. Being well-calibrated means a simple, beautiful thing - your confidence matches how often you are actually right. If you say "70% sure" about many different things, then about 70% of those things should come true. Not 40%. Not 95%. About 70%. This study is about that match. It is one of the most useful ideas you will ever learn for reading markets, because the market is full of people whose confidence dial, like Arjun's, is badly broken - and they do not even know it.

Here is the key idea, drawn as a picture. Imagine we collect all the times a person said "50% sure," all the times they said "60% sure," "70% sure," and so on. For each group, we check: out of all those guesses, how many actually came true?

A perfectly calibrated person makes a straight diagonal line. When they say 50%, half come true. When they say 70%, seven in ten come true. When they say 90%, nine in ten come true. Their words and reality march together, step for step.

Now look at the lower, curved line. That is a person who is too confident - the most common mistake by far. When they say "90% sure," the thing only comes true 65% of the time. When they say "almost certain," they are often wrong. Their line sags below the honest diagonal. They feel sure, but reality does not agree with their feeling.

Notice something important: being well-calibrated does not mean always saying 100%. A calibrated person who says "60% sure" and is right 60% of the time is doing a perfect job - even though they are wrong four times out of ten! Calibration is not about being sure. It is about being honest about how sure you should be. Saying "I am only 55% sure" can be a sign of a very good thinker, not a weak one.

For reading markets, this changes how you listen to everyone. The expert who says "this share will definitely go up" is telling you their confidence is 100%. But almost nothing in markets is 100%. A person whose dial is calibrated would almost never say "definitely." When you hear "definitely" and "guaranteed" and "cannot fail," you are usually listening to a broken confidence dial - someone who says 90% but is right far less often.

illustrative Let us test two people, Kabir and Asha, who both guess whether it will rain. Over one monsoon season, we only count the days when each of them said, "I am 80% sure it will rain today." Then we check what really happened.

Look carefully. Both said "80% sure" on twenty days. But when Kabir said 80%, rain came only half the time - his true rate was about 50%. His dial is broken: he shouts 80% when the honest number is 50%. Asha's 80% days brought rain 16 times out of 20 - exactly 80%. Her word and reality match. She is well-calibrated.

Here is the surprising part. Asha is not necessarily a better weather scientist than Kabir. She might even know less about clouds. But she is honest about her own sureness. She has learned, by checking her past guesses, what her "80%" really means. Kabir feels just as sure as Asha, but he never checked, so his feeling and reality drifted apart.

This is why calibration is a skill you build by keeping score. You cannot fix a dial you never look at. Every time you write down how sure you were and then check what happened, you slowly learn to make your "70%" really mean 70%. And once your dial is honest, your guesses become trustworthy - to yourself, most of all.

You need many guesses to measure calibration. You cannot check calibration from one guess. If someone says "90% sure" one time and is wrong, that alone proves nothing - even a 90% chance fails one time in ten. You only see a person's true calibration across dozens of guesses. So do not judge a dial from a single day; watch the whole record.

Under-confidence is a mistake too. Being too sure is the common problem, but the opposite exists. A person who says "50%" about things that are really almost certain is also badly calibrated - they are hiding real knowledge behind fake doubt. Good calibration is not about being timid. It is about matching your confidence to reality, in both directions.

A calibrated person can still be wrong a lot. Remember, if someone is well-calibrated at 60%, they are supposed to be wrong 40% of the time. Calibration does not make you right more often on any single question - it makes your stated confidence honest. Do not expect a calibrated person to be a fortune-teller. Expect their "sure" to mean something you can trust.

This idea is a powerful filter for Indian markets, and it needs nothing but a notebook. Our markets are loud with certainty. "This share will double, guaranteed." "The market will crash next month, 100%." Calibration teaches you to hear these words differently. A person with an honest dial almost never says "guaranteed" or "100%," because they know the future is uncertain. So the very confidence that impresses most people should make you more careful.

You can also train your own dial on anything around you - cricket scores, exam results, the monsoon, election outcomes. Write your guess as a number, then check later. Over a few months you will discover whether your "80%" really means 80%, and you will almost certainly find, like most people, that you were too sure. That discovery is a gift. It makes you humble in exactly the right way, and a humble, honest guesser is far safer near money than a confident one whose dial has never been checked. In a market full of loud certainty, a quiet, calibrated mind is a rare and valuable thing.

• Attach a number to your sureness. Say "70% sure," not "probably." Only a number can be checked against reality later.
• Group your guesses and count. Take all your "70%" guesses together; if far fewer than seven in ten came true, your dial is too confident.
• Be suspicious of 'definitely' and 'guaranteed'. In markets, almost nothing is 100%. Total certainty usually means a broken confidence dial, not real knowledge.
• Judge dials over many guesses, not one. A single wrong "90%" proves nothing. Watch a person's whole record before trusting their confidence.
• Fix both directions. Too sure is common, but too timid is also wrong. Aim to match your confidence to how often you are truly right.
• Keep a score to build the skill. You cannot calibrate a dial you never check, so write down guesses and revisit them honestly.