Why is this interesting? - The Certainty Edition

Hi all, happy holidays from myself and Colin. We’re hugely appreciative that you all spent some time with us this year. As for the schedule this week, we will be taking tomorrow and Thursday off (25th & 26th) for some much-needed rest and eggnog and we’ll be back in the saddle on Friday. Have a great holiday! - Noah (NRB)

Noah here. I was recently accused of being obsessed with gambling. It wasn’t from driving to New Jersey to bet on sports (though I’ve considered it), but rather because I was trying to convince a friend to make a bet on a political prediction he was making. I wasn’t pushing him to bet because I was trying to take his money (though I feel confident I would have), but rather I was legitimately trying to understand how strongly he believed in what he was arguing. When he said he was sure this particular set of events was going to occur, I didn’t know if he was saying he was 90% or 10% certain. And there’s obviously a vast gap between those probabilities. 

We’re about to enter an election year and, while we plan to avoid domestic politics, there are lots of second- and third-order effects to explore. One of those is obviously an increase in arguments about an event that’s going to occur within a specific timeframe and with a specific outcome. In other words, it’s a nearly perfect set of circumstances to bet on.

Why is this interesting?

Although betting is often framed as negative, it’s nothing more than a quantification of the odds something will occur. The line between what’s legal to bet on (the stock market) and what’s not (the NBA and NFL in all but a few states) seems more a matter of history and politics than anything inherent in the practice itself. If, for example, you wanted to bet on the Clippers to win their Christmas game over their stadium-mate Los Angeles Lakers, you can get a line of -114 (or at least you could on Sunday). That “moneyline” means you’ve got to bet $114 to make $100 or, put another way, the oddsmaker (in this case FanDuel), thinks the Clippers have a 53% chance of winning the game. (For sports betting there are other factors at play, but let’s just imagine it’s this simple for the purpose of explaining. Also, if you want to understand the math on converting moneylines to probability, it’s all here.) The point is that it’s easy to say you “think” or you’re “sure” the Clippers will beat the Lakers, but deciding to bet on them makes you quantify your certainty to above 53%.

Going back to the arguments, there are two compounding problems that happen when you talk to people about their forecasts. First, when you’re just talking, it’s easy to overstate your case. Talk is cheap, and when the goal is to win an argument people frequently come up with scenarios that sound way more likely than they actually are. Second, words are a lot more slippery than numbers. Without clarity on whether something that will “definitely happen” means it’s 70% likely or 90% likely, it’s easy to find yourself having a conversation where both parties are unclear on the points being made. These compound because when it’s easy (meaning free) to use inexact language, people will make their forecasts even fuzzier. 

Sherman Kent, an analyst for the CIA, noticed this language problem when he worked for the agency in the 60’s. When making forecasts he realized that language like “serious possibility” was included in analysis without a clear explanation of what the words meant. In a 1964 paper he told the story of realizing just how big the gap could be:

A few days after the estimate appeared, I was in informal conversation with the Policy Planning Staff's chairman. We spoke of Yugoslavia and the estimate. Suddenly he said, "By the way, what did you people mean by the expression `serious possibility'? What kind of odds did you have in mind?" I told him that my personal estimate was on the dark side, namely, that the odds were around 65 to 35 in favor of an attack. He was somewhat jolted by this; he and his colleagues had read "serious possibility" to mean odds very considerably lower. Understandably troubled by this want of communication, I began asking my own colleagues on the Board of National Estimates what odds they had had in mind when they agreed to that wording. It was another jolt to find that each Board member had had somewhat different odds in mind and the low man was thinking of about 20 to 80, the high of 80 to 20. The rest ranged in between.

In response, Kent suggested the agency adopt a shared vocabulary of forecasting. The table attempted to associate numerical probabilities to the various words that might occur in forecasts. Here’s what the table looked like:

In the end, the agency never adopted the approach. As Philip Tetlock explains in his book Superforecasting:

But it was never adopted. People liked clarity and precision in principle but when it came time to make clear and precise forecasts they weren’t so keen on numbers. Some said it felt unnatural or awkward, which it does when you’ve spent a lifetime using vague language, but that’s a weak argument against change. Others expressed an aesthetic revulsion. Language has its own poetry, they felt, and it’s tacky to talk explicitly about numerical odds. It makes you sound like a bookie. Kent wasn’t impressed. “I’d rather be a bookie than a goddamn poet,” was his legendary response.

So, as you find yourselves in various arguments in the coming year you might want to try moving to bets instead of words. In that case, the odds and commitment to the argument should become clear. In the case where you can’t convince someone to bet, at least you can pull out the Kent chart and ask them to clarify which probability they really mean. (NRB)

Book of the day:

If you’re in a bind and need to get one last present, may I suggest Kevin Wilson’s new novel Nothing to See Here? I put it on hold at the library after reading what must be one of the effusive first paragraphs of a book review that exists. Here’s how Taffy Brodesser-Akner opened her review: “Good Lord, I can’t believe how good this book is. I know you’re supposed to begin book reviews with subtlety and a nod to storytelling’s past and the long literary tradition that the book has managed to hook itself onto. But ‘Nothing to See Here,’ the third novel by Kevin Wilson (‘The Family Fang’), defies an entry like that because it’s wholly original. It’s also perfect.” I ripped through the novel about two kids who self-combust in just a few days and it was as delightful as the review suggested. (NRB)

Quick Links:

• The art of dying (CJN)

• Chat app or surveillance tool? (CJN)

• Honestly, a Video Game That Strands You On a Boring Six-Hour Flight Is Just What I Need Right Now (NRB)

Thanks for reading,

Noah (NRB) & Colin (CJN)

PS - Noah here. I’ve started a new company and we are looking for a sr. backend engineer to join the team. If you are one of those or know anyone great, please share. Dinner’s on me at a restaurant of your choice if you help us find someone.

Why is this interesting? is a daily email from Noah Brier & Colin Nagy (and friends!) about interesting things. If you’ve enjoyed this edition, please consider forwarding it to a friend. If you’re reading it for the first time, consider subscribing (it’s free!).