The Sales Hacker Podcast
The Sales Hacker Podcast

Episode · 3 years ago

31. Annie Duke, Author of "Thinking in Bets" and Celebrity Poker Player

ABOUT THIS EPISODE

This week on the Sales Hacker podcast, we are delighted to speak with world-famous author and teacher, Annie Duke. Annie is a former World Series of Poker winner and has written an incredible book detailing how thinking in bets can improve outcomes. In this episode, we take those concepts and apply them to sales.

One, two, one, three, three, hey everybody, it is the sales hacker podcast. It's yourhost, Sam Jacobs. I am the founder of the New York revenue collective. I suppose I'm chairman of the global revenue collective. I can give myselfany title I want when it comes to the revenue collective. I'm also thecrow at a place called Behaveos, which is a big day of machine learningplatform. And today we've got an incredible interview because we've got any duke,the world famous poker player. She won the world series of poker a fewtimes. She was on the celebrity apprentice. So she has experienced with our wonderfulpresident and she's also exceptionally insightful and she's written a book called thinking andbets and I encourage everybody to read it. We talked about that book in thisepisode. So before we get into the episode, let's thank our sponsors. We've got to the first is air call. It's a phone system designedfor the modern sales team. They seamlessly integrate into your crm, they eliminatedata entry for your reps and they provide you, if you as the manager, with greater visibility into your team's performance through advanced reporting. I when it'stime to scale, you can add new lines and minutes and you can useincall coaching to reduce ramptime for you new reps. so that website is arecalled that I owe for its sales hacker. That's once again are called DOT IO. Forward sales hacker to see why Uber, done and Bradstreet, pipe, drive and thousands of others trust are call for their most critical sales conversations. Our second sponsor is outreach, dot Io, the leading sales engagement platform. Outreach triples the productivity of sales teams and empowers them to drive predictable andmeasurable revenue growth by prioritizing the right activities in scale and customer engagement. Withintelligent automation, outreach makes customer facing teams more effective and improves visibility into whatreally drives results. Hop over to outreach that I oh forward sales hacker tosee how thousands of customers, including cloud era, glass door, Pandora andZillo, rely on outreach to deliver higher revenue. Her sales up. So, without further ado, let's listen to the interview with Annie Duke. Highfolks, it's Sam Jacobs, and welcome back to the sales hacker podcast.Were incredibly excited for our guest today, Annie Duke, who is an awardwinning author, Professional Speaker, decision strategist and also former professional poker player.Annie has just released a book, or I guess, earlier this year.The book is called thinking and bets, making smarter decisions when you don't haveall the facts. I first found out about the book through mark and reasonand his twitter feed and then I downloaded it, read it and was itjust really, really intrigued by the ideas present in it, and so that'show we got Annie onto the show right now. Let me just tell youbriefly about our background. For two decades and he was one of the toppoker players in the world. In two thousand and four she won the worldseries of poker, her first bracelet. She also won a big tournament thatyear in the Tournament of champions. In Two thousand and ten she won theprestigious NBC National Heads at Poker Championship. She's also, by the way,a student of cognitive psychology, so she was also awarded a National Science Foundationand fellowship to study cognitive psychology. At you pen. She now spends hertime writing, coaching speaking on a wide variety of topics. She's regularly soughtafter as a public speaker. She is a philanthropist. She's written other booksabout poker mostly, and this is sort of I think, but she'll correctme if I'm wrong, one of the first books that she has it's reallynot about poker at all but just about decisionmaking. She's a master storyteller.It works. Incredibly excited to have her here on the show. So welcomeAnnie to the show. Thanks for having me, Sam. We're excited aboutit. So the context for this conversation is I bought your book. Iread it. I thought it was really, really interesting and I think it's incrediblyrelevant to the sales profession. But first let's just hear a little bitabout your background again. We know you, I think, is now a professionalspeaker and decision stratagist, as you say, and a coach, butI think a lot of the world knows you as a poker player. Wheredid you come from? You know where to grow up. You talk aboutthis in some of the book, but walk us through how we got herea little bit. We're okay. Well, I was bored in a small campsure. Well, I mean I'll start with my adult life. I startedoff doing my Undergrad work at Columbia. I went into major actually an Englishlit and but ended up double majoring in English and psychology, partly because ofthe mentorship of wonderful professor. They are named Barbara Landau and I was actuallyher research assistant for four years and she was working in the Psyche Department thereand she encouraged me it took on to Bradwood School, which I did.I got into the University of Pennsylvania Program and got, as you said,in National Science Foundation Fellowship to study cognitive science and I was working with LylonHenry Gliteman, who actually the book that I wrote is dedicated to they're reallymy intellectual parents and most ways, and so I was studying there and Iwas there for five years and did all the things you're supposed to do to, you know, get your PhD. I studying in specific first language acquisition. But what kind of psychology is really dealing with is how do you interpretthe world around you? Right? So...

...they're dealing with questions that kind ofcross with a lot of different fields and disciplines. You know, biology,physics, physiology, a little bit of philosophy, business, psychology and it'ssome computer science, and that's really put together into this kind of cross disciplinaryview of the way that we interpret the world. So a lot of whatI was thinking about in particular was learning and how you learn, and inmy particular interest was and how you learn a first language. So I'm goingalong and right at the end, so this is after five years, I'mgoing off to interview to become a professor for my job talks and academics,and I had been struggling that year with some stomach issues that got really,really bad right around the time that I was supposed to be going out formy job interviews and actually on my way to Nyu for my first job talk, I got so sick that I landed in the hospital and I was therefor actually two weeks, and so I had to delay going out onto thejob market, and in academics that market is seasonal, so I couldn't justdelay it a few months while I got better. I had to delay ita whole year. So during that year when I was recuperating, I wasn'tteaching, which was one of my sources of income. I certainly didn't havea fellowship. So I just found myself in need of money, honestly,and my brother, Howard letter, had already started playing professional poker at thattime, through somewhat circuitous route, if you we can talk about that ifyou want, but he had already found his way to poker, which isvery unusual at that time because poker was not on television. It wasn't somethingthat as a kid you could just sort of like see, oh, thisis something that I want to do. It wasn't really a known profession,but he had found his way there and so I was aware of it andhe suggested to me, Oh, since you need money, this would bea good thing to do in the meantime and I'll teach you how to play, like you've watched me play some and I'll teach you how to play.So it was during that recuperation time that I started playing and I thought,okay, I'll do this and then next year I'll go back out on theacademic job market. But I ended up really taking to the game. Ireally, really loved it. I love the problem that it was presenting tome and you know, in retrospect, I now realized that I didn't actuallychange what my area of interest was, because when I was playing poker,what I was really trying to figure out was, you know, i.How are you learning this game where you getting tripped up? How do youdeal with this very noisy feedback that you're getting from the way that things areturning out? How are you mapping out the future when it's probabilistic? Youknow, all of these kinds of questions that were actually very similar to thekinds of questions that I was asking the graduate students. So essentially, themeantime turned into like twenty years, eighteen years or twenty years, and thenI retired in two thousand and twelve. And this was all in Montana.That's I mean, I think, an interesting wrinkle right, if I'm notmissed. So it started in Montana. So I was I was obviously inPhiladelphia when I was at you pan, and when I was recuperating, mythen husband actually had a house in Montana. His family lived in Montana, sowe just decided we would go out there. Well, I was tryingto get better instead of staying in Philly in my apartment there and, youknow, me continuing to do research that it would be better if I wasreally trying to take some time off. So that's how I ended up inMontana and I was there for about three years. That was really where Iwould say my apprenticeship occurred, and by the time I was done there Ihad declared myself professional. I had already cash in three different world series ofpoker tournaments by that time and I'd actually made a final table and come insecond one. And so at that point I moved down. I'd made thedecision to move down to Las Vegas and pursue it full time. So thatthat was in one thousand nine hundred and ninety four that I decided to pursueit's full time and become a professional. Then the real kind of Aha momentfor me, I think, was in two thousand and two. So nowthis is eight years sin too my professional life as a poker player. Afriend of mine named Eric Seidel, who has mentioned quite heavily in the bookas you know, who really I consider one of my main mentors in thegame. He had started his life off partly as an options trader and he'dbeen trading on the floor in New York and also I think in La andhe had made. He had developed obviously some relationships with that and one ofthose relationships that guy named Roger Lowe, had asked him if he would comeand speak to retreat of options traders. So Roger Lowe had a hedge fundat that time and had a group of options traders and he wanted Eric tocome and talk to them about how poker might inform of your decisions about tradingand risk. So Eric is notoriously shy about public speaking. He really doesnot like getting up in front of large audiences and talking. so He'd demurred, but he said well, maybe you should have anny do it, becauseAnnie actually used to he shoes to teach...

...college. So I assume she's prettygood in front of an audience since she has a lot of practice at it. So I got asked to do it and it was it was really actuallythe first time that I thought in a very explicit way about what the relationshipbetween my academic life was and what I've been doing it at poker and reallystarted to explicitly figure out how can I express this relationship between these two thingsand what poker might teach you about the challenges of learning and how we're thinkingabout the future, how we control our emotions when we're in the moment ofa decision, what kind of biases are really tripping us up and preventing usfrom really experiencing the learning curve that should be available to and poker because theamount of feedback that you're getting. So it really brought to light at thisthis really big problem that I sort of realized that I'd been trying to tackle, that I've been thinking about a lot as I was learning to play thegame, which is that in graduate school, you know what I had been taughtwith that learning occurs when you have lots and lots of feedback tied closelyin time to decisions and actions. It's very common thing. You hear itin every single psych one class and I thought, well, this is interesting, because here I've come into this game. You know, when I was playingin Montana and I was playing against people who have been playing the gamefor decades, and you know, certainly I did. I did have avery good teacher and my brother at that time, but certainly did not havethe experience. I did not have the number of hands out of my beltthat these guys had, and honestly, they should have been a lot betterthan I was, you know, just given the sheer amount of experience thatthey had had. But yet they weren't. And you know, I didn't thinkit was something particularly special about me. I thought it was something special aboutthe problem that poker was presenting in terms of how you actually learn fromwhat's happening around you. And what I realized was that this idea of,you know, learning occurs when you have lots and lots of outcomes tied closelyin time to decisions and actions, is missing a sentence, which is exceptwhen the feedback is really, really noisy, and what that means is when theactual outcomes that you're getting are only very, very loosely related to thedecisions and actions, which is exactly what's happening in poker. So over thelong run in poker, if I'm playing really well against you, I'll endup winning over the long run. Likewise, if I'm playing worse than you,over the long run, you know that difference is going to reveal itself, but it takes a while. So on any given hand, or evenon any given, you know, eight hour session that I might play,I could end up winning even though I've played quite well in comparison to you, or I could end up losing. I could end up winning even thoughI quit played quite quite poorly, rather in comparison to you, and Icould end up losing even though I bet I've actually played quite well in comparisonto you. So this present a really big problem because whether you've played thehand well or not is kind of hidden from you because I can't see theother players cards, so I don't really know what they're holding. And evenif I could see the other players cards, I still have to guess about howyou might react to them. So I have this sort of two stageprocess, which is guessing at what your whole cards are, which are faceddown to me, and then also trying to figure out how you might reactto those cards. Are Certain Bets, and at the end of the handthat stuff is mostly never revealed. So I'm just taking the outcome I wantor lost the hand, and I'm trying to work back to uncover this thingthat's actually pretty opaque, and so it's very noisy because it's not a perfectsignal and the way that you can sort of feel like, okay, it'snot a perfect signal is to think about it in comparison to a game likechess. So in chess, if I lose to you, like, ifI play a game of chess against you and I lose, like, whatdo we know about my decision making in comparison to your? Yeah, well, we obviously know it's a very discrete outcome. So we can say itwas either worse or better. And and I think the other point is thatthere's always a I mean the maths would say that there's always a correct decisionin chess, correct and there's probably not in poker. I would guess.Well, there's a theoretically correct decision in poker if you could turn poke intoa game like chess. So you know, we could think about that comparison.Right. So in Che the reason why, in chest there's a theoreticallycorrect answer that we can go back and work out, and the reason whywe know that, if we if I have a bad outcome, that Ican work backwards and say that I've made worst decisions than you, is becauseyou're losing a lot of this noise, right, a lot of the uncertainty, a lot of this sort of probabilistic nature of things when you're playing chessand you're losing it for two reasons. One is that there isn't a superstrong luck element, meaning somebody isn't randomly coming across and like taking, youknow, a pawn off your board or something at like random intervals where youdon't know when that's going to happen, which is a little bit what happensat poker when and there's a bad turn of the card. But also thereisn't a hidden information element, so there's...

...no information asymmetry. I can seeyour position just as well as you can see my position. Yeah, andwhat that means is that we can actually work it out. So as aplayer, I can say, like I can see what all of your possiblemoves are and I also know that the the pieces are only going to moveon purpose, right, because someone does something on purpose to make the movethat way. So it just presents like a super different problem that takes thenoise out of the problem in a way where you can actually work backwards andtry to figure out, you know, why you want are lost, andwhat that means is that learning proceeds at a much more orderly path in agame like chess. Now chess is very complex. So it's hard, youhave to it's a lot to master and most people, obviously, you know, aren't going to master it. But the learning proceeds more orderly and generally. You know, someone who had been playing chess really seriously for twenty yearswould always crush someone who came in essentially on the first day, and that'snot necessarily the case in poker. So poker really presents a unique problem andthe thing that I think I found really fascinating was it's the unique problem thatI was studying when I was in graduate school, because poker looks a lotmore like the decisions that we make in life. tegging it back both tofirst to the audience, which is sales, and sales by definition is probabilistic,you know, and our win rates from this maybe to inside baseball onsome of the jargon, but you know, we create opportunities that sort of likethe notion of a contain sales conversation. Those opportunities should win on the lowend, depending on how you define it, maybe like eight to tenpercent of the time and then on the very high end, from two thousandand twenty five thirty percent of the time. So either way we're losing most ofthe time. Yep, and similarly, I mean similarly in Poker, I'msure you you lose more hands, or at least you fold more hands, than you win, and you have to know how to size the betand do all of those things. But one of the key concepts in thebook that you articulate is this concept of resulting. So walk us through youknow like what that means, because I think it's and we were talking aboutit, you know, offline before, but it's probably like the most prevalentthing that happens, you know, certainly in sports, but really in life, which is so tell us what the definition is? Yeah, sure.So resulting is essentially working backwards from the quality of the outcome to the qualityof the decision. So it's basically saying, if I can see what the qualityof the outcome is, then that tells me what I need to knowabout the quality of the decision. Now, resulting in something that's particularly a problemwhen you're trying to evaluate what somebody else is done who you aren't indirect competition with. So this is why we see it so much in sport. So the example that I open the book with obviously is Peak Carol,the Seahawks, two thousand and fifteen super bowl forty nine, last play ofthe game, and so they're against the New England patriots at second down.They're down by four, so they need to score a touchdown here and they'reon the Patriots one yard line and they have one time out left. Sothis is like one of the most amazing examples of resulting that that I've everseen in my whole life, which is why I open the book with it. So basically what happens is that Pete Carroll is expected. There's an expectedplay here, and they expected play is that pee carrol's going to hand itoff to their running back, marsh on Lynch. Well, he's moll isit personally going to hand it off? He's gonna have Russell Wilson handed up. That's right. Yeah, and and Marshawn Lynch is one of the bestshort yardage running backs in the history of the game, which is I need. Actually he had had a great game and, you know, run througha bunch of Patriots earlier. So right, exactly. So everybody thinks, okay, he's going to hand it off to Marshall Lynch, that's the obviousplay. Marshall Lynch is going to try to get the one yard. Youknow, obviously if Marshall Lynch fails there, they have this time out in theirback pockets, so they'll call the time out and they'll take another shotat it, and that that's the way that they think that the game isperceived going to proceed. But Pie Carroll actually does something super surprising here.He has Russell Wilson actually passed the ball. Russell Wilson does that. He passesthe ball and it's pretty famously intercepted by Malcolm Butler and obviously that endsthe game and I really highly recommend that people go and try to find thefootage on Youtube because I want you to listen to Chris Collingsworth call this andyou can hear Chris Collingsworth just just you know, this is the worst placeever seen. Basically, like I don't know what he was thinking. Thisis so ridiculous. Why did he make this call and then the next way, even after people had had time to think about it? So, youknow, in Chris Collins Worth Defense he's making this judgment obviously on the flyright then. But the people who are writing it about it the next day. I have had a lot of time to think about whether this play wasany good or not. But they didn't really say anything different than Chris collingsworth said, and there seemed to be kind of a the only disagreement amongthe major newspapers was was it the worst play in Super Bowl history or wasit the worst play in NFL history period? Now when I think is really interestingabout this is that what we do know is that it was definitely oneof the worst outcomes in Super Bowl history,...

...that's for sure, and for sureit was one of the worst outcomes in NFL history, although I wouldargue like that Joe, thisman getting his leg broken thing, which that wasprobably worst outcome, at least for Joeth Siseman, but and for Washington Redskinsfans, of which I am one. Yes, exactly, but but youknow, it's certainly it's going to be competitive here for one of the worstoutcomes in Super Bowl history. But that does not mean that it was sucha terrible decision as everybody said. And there were a couple of voices thatwere arguing a different stance. One of them was Benjamin Morris on thirty eight. Another one was, I think, Brian Butler on slate. You cango read them if you want, but basically the point that they were makingwas whoa hold on a second guy's don't you want to kind of know whatthe interception rate is here, because if it's super low, maybe we shouldstart to think about what was Pete Carroll getting in exchange for the play?So if we start they're a super conservative estimate of how often that ball isgoing to be intercepted. There is around two percent. The range of estimatesis going to be between one and two percent. So we already know thatthis was a really, really low probability event. So let's just start there. And then the second really key piece of information is that if you handit off to Marsha on Lynch, remember they only have twenty six seconds left. The only way to stop the clock if Marsha on Lynch fails is tocall the time out, and by the time that that has occurred a lotof time will have rolled off the clock because obviously a run play takes longerthan a pass play and by that time they're only going to have a chance, one more chance at the end zone. But if they pass the ball there'sreally kind of three outcomes that occur. We're setting aside fumbles, which weshould, because you can fumble also if you're running the ball, andthe three outcomes are an interception, which is what happened, which we knowis only between one and two percent of the time, an incomplete pass,which stops the clock very quickly Aha and allows you to run the ball twiceagain, or a touchdown, which obviously wins the game. So that's thatkey piece is that when the ball is incomplete, the clock stops so quicklythat Seattle can still hand it off to marshawn Lynch twice. So basically whatthey were arguing was for this little like one to two percent chance that thisdisaster happens, what you're getting in exchange is three tries at the end zoneinstead of two. You still get those two running plays. If you happento fail on the past side, almost all the time that you fail,you still get the too run play. So you know, you can agreeor disagree with that. Like I'm willing to have a conversation with you about, you know, where the stakes too high or, you know, wasthat too high of an interception rate to Willie with and but I think thatwe can get to really understanding what's going on here, that they were reallyover lying on the fact that the outcome was so bad. But just askinga simple thought experiment, and the thin simple thought experiment is, what doyou think would have happened if the ball was caught for a touchdown? Whatdo you think Chris Collingsworth would have been saying in game and what do youthink that the headlines would have read the next day? Now, brilliant call, of course, yes, and actually I mean, just to sort ofprove the point, you can go listen to Chris Collingsworth call the PHILLY specialfrom this year Super Bowl, which was an incredibly unexpected play. The Eagleswere on the Patriots one yard line again and it was fourth down and theeagles were up by two at the end of the second by three rather atthe end of the second quarter, and everybody expected they were going to goor field goal and go into the locker room up by six. And instead, not only did Doug Peterson not put out the kicking team, but insteadhe went for it and Nick foles went around and caught the ball in theend zone somehow. So the quarterback ended up in the end zone as areceiver. Now, that play happened to work out equally unexpected and you know, Chris Collingsworth was calling him a genius and game in the next stay.Of course, that was all the talk of the newspapers about how smart DougPeterson was. So we actually have an example of what happens when you havetwo kind of equally weird plays and in one case it works out and inone case it doesn't. And I think that we can also do the reversedot experiment on the Philly special and say, well, what if that ball hadbeen intercepted, they had not gotten the three points, you know,in the Eagles had gone on to lose the game, no doubt people wouldhave been going and pointing at that particular play. So what we can learnfrom this is that we really make this really big mistake, which is wethink, you know, if we have a good outcome, if we closethe sale, and we obviously had a brilliant process and we should try torepeat that, we should be, you know, patting ourselves on the backfor how great a job we did. And if somebody, if we seesomebody not close a sale, if someone's working for us or whatever it mightbe, then we're much more likely to sort of point fingers and try todig in and look at where the mistakes were and where the blame was.But what we know is that things that turn out badly aren't necessarily just becauseof mistakes, and things are their turn out well aren't necessarily a mistake freeyeah, I mean, I think exactly...

...to your point. The last thingon the football part is that I think the the statisticians have done the mathand it. I think the result is that, like in almost every situation, very rare, I say the majority of situations, people should go forit on fourth down and they never hurtetting, by the way, on your ownone yard life. Yeah, exactly, but they never do right. So, yeah, and actually, so the reason why I should go forit on your own one yard line, just for people who are a littlebit more statistics x Geeks, is because when you punt, the other teamalmost always ends up inside the fifty and so they're already kind of guaranteed threepoints anyway. Yeah. No, so you're already giving up three points andso you're just supposed to try to go for it on and the one yardline and on your own one yard line on fourth down and just try toget the first down in order to move the ball a little bit farther downthe field. The fascinating thing is the the sports journalism world, in themedia, because they are so heavily focused on outcomes, they'll construct entire narrativesout of, you know, the Grit and the virtue and the determination.Those are all adjectives that are going to be used on a probabilistic outcome whereyou win and then if you again make the same decision and you lose,then it's something about your spirit and you're no, you're not strong enough andthere's weakness and the coach wasn't good. So that's exactly that's exactly right.That what's happening is that on the downside, people are looking for things that werein control of the team, right in the control of the coach,that, you know, they just didn't have the spirit to win or,you know, they weren't good enough to overcome or, you know, whateverit might be. And also on the upside, they're also saying, oh, that was because of their grit, but we know there's a lot ofrandomness right, like if you think about basketball, for example, when you'retalking about like a one point lots or one point win, a lot ofit just has to do with sort of randomly, you know who ends upwith the ball last. Yeah, I mean, I mean I'm a hockeyfan, so there's nobody that knows about probabilities more than a questioning capitals hockeyfans. And, by the way, so speaking of the fourth down thing, the NHL is another lead that really has this problem, because there's beena lot of math done on when you should pull the goalie and people shouldbe pulling the goalie a lot earlier in the game than they actually do.And why don't they do that? Well, because they know that the fans andthe ownership are all results. I mean this is a problem, right, like Pete Carroll got so excoriated when this didn't work out well, andthat happens to NHL coaches as well. If they pull the goalie in aplace where people don't kind of sort of accept that that's maybe something that youshould do, then if they go on to lose the game, what's goingto happen? Like people are going to point at the coach and say,I can't believe how bad they are. Oh, clearly, you know.Clearly there's such a failure. And here's the problem. So this is wherewe can really kind of tie these together and see how this really affects behavior. Is that now people see the treatment that Pete Carroll gets. People seehow, if you pull the goalie, you get completely yelled at. Youknow, people see what happens when you go for it on fourth down andunder unexpected situations and you fail, what kind of treatment the press gives youand the fans give you. So what does that drive people to do?It tries people to not make those decisions. So even though the math is very, very clear on these kinds of things, the NFL has been incrediblyslow to adopt these fourth down plays and the NHL has certainly been incredibly slowto adopt pulling the goalie relatively or early in the game. So why isthat? Well, because they're afraid of basically getting yelled at. So wecan sort of go back to this in terms of what's happening, for example, in your world in terms of sales. So you have this world where yourprospecting and some small percentage of those prospects are going to actually realize,and the idea is that the ones that do realize are going to pay youenough in order to sort of compensate you for all the small fishing lines thatyou're putting out there. Right. But if I know that the person thatI'm reporting to is going to result on me, which is what most leadersdo, right, most leadership results, and if they didn't close the sale, they say, why didn't you close the sale, like, what didyou do wrong? And I know that that's going to happen, then whatis that going to drive me to do? It's going to drive me to tryto close the sale under kind of under all circumstances. So what I'mnot going to do is tend to push the boundaries. In other words,it's going to drive me to try to close as many sales as possible sothat I can sort of fend off this resulting problem. I can spend offthis kind of like when I don't close, I'm going to have this deep divecome in on me. Right. So it's going to cause me tomaximize the number, not maximize the amount, and that's obviously really bad if peopleon your team are behaving in that way and if people on your teamare behaving in that way where they're trying to maximize number as opposed to maximizeactually how much they close the sale for. Generally, if people are behaving thatway, it's your fault because you're...

...acting like the fans of the seahawks. Yeah, I mean then there's a again, whether it's sales or anyother aspect of life, people fundamentally misunderstand like the notion of expected value.Right, I have a question for you in my world. So my worldis enterprise sales, and these are twelvemonth, sometimes eighteen month sales cycles and oneof the things that you mentioned poker so useful for is because you're gettingfeedback all the time on decisions that you make that are determining outcome. Soyou have enough data to analyze your process and to make improvements. How doyou deal with the situation where these are binary outcomes? You know, weeither close wheel or we don't, and for very large amounts of money.So they expected value is fantastic, but the forecasting of these deals is verydifficult and also there's this resulting problem where I'm not getting enough. I don'thave that much data for me to in any constrained a period of time,particularly it startups, because, you know, we're trying to like double and tripleand twelve month periods, and so how do you deal with the situationwhere you're not getting enough feedback, there's not enough signal coming back to youin order to, you know, adjust and improve the process? Yeah,so that's a really interesting question. So let me just roll back sort ofa little bit earlier and what you said to say it in some ways thatslower cycle is actually advantageous. So let me just explain why. So oneof the problems of poker is that you're getting so much feedback that the pastthat you're on becomes very, very clear to you. In other words,you know, you're sort of getting pounded by feedback and people aren't natural dataaggregator. So it's not like I'm getting a series of results and I'm sayingwe'll let me wait and step back and sort of take the average of theseresults and kind of way to make a decision about what each of these outcomesmeans. And I'm going to step back and I'm going to wait until theend of the session to actually go back and try to figure this out.Instead, what we tend to do is we feel outcomes in sequence. Sowe feel them one at a time and the decisions that we've made about eachoutcome and affect the kinds of ways that we think about the next outcome.And only that you can see really clearly when you're sort of on a downwardtrend versus and the upward trend, and the way that we actually view what'sgoing on around us is very path dependent. So the way that we feel aboutit, the way that we think about it, the way that weanalyze it is very path dependent. So you're right, there's a lot ofthere's quite a bit of upside to poker in terms of the shortness of thecycle. You get feedback off your decisions within, you know, thirty seconds, and so that's really great to be having such a short cycle and it'sreally great to be having so much feedback come and such a short period oftime. But there's also a dark side to that, and that dark sideactually can cause people to get so sort of emotionally hot and be unable toeven step back and aggregate those results in any kind of way that they canactually learn from it. It's part of the reason why somebody could be playingfor twenty years and continue to make the same mistakes over and over again,partly because of the amount of data. So there's kind of good and badit. So when you have a slower sales cycle, obviously there are challengesthat come with that just in terms of your kind of fishing around in aspace where you're not closing the gap that quickly, and so it feels likeyou don't have as much certainly it's true that you don't have as much datato work with in order to make your guesses and it's a slower process,but you do get a little bit out of that path dependency. It doesallow you to approach it in a little bit more of a rational way.So I just want to just point out that there are some good things tokind of slowing it down. Yeah, yeah, we're going to say,oh no, I go ahead because I was just going to move on tothe next point. So go ahead. The next point is probably the interestingone. Go for it. So, but obviously there are downsides to nothaving as much data as well, because you know now, as you're tryingto make your estimates of what the probability is that something's going to close,you're sort of casting about a little bit more and your confidence intervals are goingto be much wider around those estimates. And then very often the answer itselfthat comes in it's very remote from a lot of the initial decisions that youmade, which actually increases the amount of luck in the process, because youknow, you make a decision to pursue a particular lead and try to closea particular deal and it might be eighteen months before you actually get to theresults. In a lot of stuff can happen in eighteen months that you literallyhave no control over. Not only that, within that eighteen month period there canbe a lot of information that reveals itself that you didn't have access toat the time of your initial decision, and so that can make it hardto sort of sort out what should I have known or what could I haveknown, you know, versus what was reasonable for me to know what wasonly going to reveal itself after the fact, after I'd already started down the path. So that can become really difficult. Yeah, I mean, I thinkthe one of the challenges is that the time interval that we're all beingjudged on, especially in sort of high growth startup land with these sales cyclesis out of sync with the length of the sale cycle, because I'm I'mjudged on a time interval that doesn't give me enough data to really build anefficient decision making process around these types of deals. Right. Yeah, sobasically what I would say about that is...

...that try to look to the goodin this, which is this that one of the ways that I recommend thatpeople deal with this outcome by us, with resulting is to actually, whenyou're deconstructing the decision as much as possible, try to deconstruct the decision process priorto the outcome of ever having occurred. And if you can't do that justbecause of time constraints, then you should deconstruct the decision making process withpeople who you don't tell the outcome too. So in that sense it's actually kindof good news that your sale cycle is so long. But you haveto get buy in from the people around you. In other words, youhave to do a lot of really good work to work through what strategy you'regoing to use and make these reasoned estimates as to what the expected value isof that particular opportunity. And once you've made that reasoned estimate of what thatexpected value is. Then everybody has to buy into it and understand. Andthe reason why it's really important to actually make these expected value estimates explicitly isbecause in making those expected outcome estimate, you have to actually go through whatthe set of possible outcomes are. Right. So if this could in this casewhere it sounds like it's binary, it's either yes or no, right, it's either Yes for this amount or no for this amount. Yeah,I mean to not to interrupt you, but even just to underscore your point, it's actually not quite binary because it's yes or no, but it's reallyoftentimes it's no, which means not right now. It's only binary against aspecific time constraint. It's not generally binary. Right. So let's talk about it, but as a specific time constraint, just to simplify it, because obviouslywe can do an expected value calculation that includes that has a conditional probability, which is no. But what's the probability that it will occur at alater date? Yeah, right. So let's take the simpler problem of it'syes or no and in any case it's always for the same amount. Right. So, in order to come up with an expected value. There youhave to really think about what's the probability of yes and what's the probability ofno, and you have to do that in an explicit way. And ifyou actually work through that process with a good team, where you're coming inwith your own estimate, hopefully you're not telling them what your estimate is,as you're trying to get at their opinion about what their estimate on that is, because all of you have different experiences and different ideas and you're all thebest experts in estimating the thing that you're trying to estimate here. Right andthere isn't a right answer, because it is probabilistic in nature and what you'retrying to come up with is your best guess. And once you've bought intothat in advance, what tends to happen, because you can memorialize that and say, okay, for Opportunity A, here's how often we thought it wouldbe as, how often we thought it would be no, and here's theexpected value on it. For Opportunity to be the same thing for opportunity seefor Opportunity d. Now there's all sorts of really good things to come outof that thing. Number one is because you've actually worked through that and gottenby and on it, hopefully, because we've worked through that as a grouper, or you've actually worked through that with the person that is judging you,so they can see what you're thinking processes and that they've made any tweaks thatthey think should go into that right. So they've sort of been wrapped intopart of the process and so they have a little bit of ownership over itas well. Now, if it's yes, hopefully you're not padding yourself on theback too much because it was in the estimate already of yes, andif it's no, hopefully you're not getting judged too much for it. That'snumber one. Number two is because you're actually actively trying to make these estimates. It starts to get you to be information hungry. So one of theproblems that we have, if we go back to that initial description that Isaid of poker, is that we've got two sources of uncertainty. One ishidden information, there's just stuff that we don't know, and the other isluck. So in order to make a really good estimate of how often youthink it's going to be yes and how often you think it's going to be. No, you have to actually really try to uncover those two things.So first of all, you have to think really clearly about how much luckmight be in the process, like, even if I do everything exactly right, right, even if I don't make a single mistake, even if Ihave all the information I need in order to come up with a perfect strategyand deploy perfect tactics and trying to execute on this, how often is thisjust out of my control and I have no say in the matter? Soit gets you to think about that really clearly in advance. But the moreimportant thing that it does is that, in order to come up with areally good estimate what you think those probabilities are in the future, you mustbecome hungry information from information, because you're trying to get those to be moreand more precise, because you're trying to model those outcomes as accurately as possible. In order to do that, you're trying to uncover really good inputs intothat model. So you're trying to figure out, like, what information amI not thinking about? What am I missing? Here's a really important one. Why might I be wrong? which is not something that we tend toask ourselves if we're not thinking about these...

...explicitly, if we're thinking supplicitly,we tend to ask why are we right? When we're actually trying to create anaccurate model of what's going to happen in the future, we start toask ourselves, why are we wrong? You're going to start to seek outopinions and you're going to be especially appreciative of dissenting opinions that actually help youto calibrate what your estimates might be. So get you to really focus onthe uncertainty, because what you're trying to do is narrow down your own kindof cone of uncertainty. Right. You're trying to make that on error andit gets you to start to uncover this stuff. Yeah, and then thelast thing it does is kind of the other after the fact of fact thatit has that's really, really good is not only are you less likely toget resulted on, both resulting on yourself or having someone result on you becauseyou bought them into the process, but now, when you go back andyou're trying to get feedback from the leads that you were working with, insteadof just going and asking the people who said no why didn't can you justgive me some feedback as to why I didn't get it, which is whatour tendency is. You're also going to go talk to people who said yesand say, Hey, can you just tell me, like, what isit about what I did that caused you to say yes? What actually doyou think I've made a mistake on? What could I have done better?What do you really think the probability of yes was where you like always ayes and no matter what I did, I was going to get a yes. was there something in particular I did that caused you to say yes,even though I did that? Were you on the fence or was it awide margin? And those are the kinds of questions that we tend not toask when we actually get a yes, we tend to just sort of leaveit and say, Yay me, I'm so excited, like look at me, I'm so excellent. But the problem is that it is actually really importantfor your ability to estimate properly in the future to know was it a narrowyes or was it a wide yes? Was it something you did or wasit yes that you were right place, right time, like, and thereason why we don't like to go in and sort of Dick take a deepdive on this and explore it. Is because one of the things that wereally like to do is get those pad on the back of moments, right, to get those moments of feeling a lot of credit. And if wego in in the person tells us like, actually, it was really close andI just flipped a coin, it feels like we're sort of losing creditfor own part in whether that closed or not, because it feels like badnews then that it wasn't wide and it wasn't like I was so brilliant thatit wasn't even close. But the thing is that when you actually refocus onmy whole goal is always to be the most accurate estimator, like that's whatI really care about, then that then becomes your reward and the fact thatyou're even willing to go in and talk to somebody and get some bad newson something that you thought was good news becomes what becomes sort of the selfreinforcing action. And the other thing is that when we actually try to thinkit expected values this way. The other thing is that we actually get much, much better, it's at creating a good work stack right, like weunderstand what the higher priorities are. So it's an example. Like I wasworking with a nonprofit that was, you know, obviously has a lot ofgrant prospecting happening, and so it's a similar situation to you. It's verylong cycle and it's either a yes or no. Right. So there's someaward amount on the grant, it's a hundred thousand dollar grant. They're eithergoing to get a yes or no and it's a long cycle decision. Sowhat was happening when I came into consult with them is that they were naturallystacking the highest dollar amount, like the highest award amount grants, at thetop, and that was the thing that they were spending the most time on. It was certainly what they were putting the most senior people on. Ifthey were hiring an outside grant writer, you know, a contractor to comein to work with them, they were putting in them on these tie dollaraccount and when I had them come in and start doing this expected value work, what ended up happening was a much more efficient stacking because if you havea hundred thousand dollar potential award that you're going to get twenty five percent ofthe time that is actually the expected value on that is twenty five thousand dollars. But if you have a fifty thou grant that you're going to get seventypercent of the time, that's worth thirty five thousand dollars. So what happensis that you start to see that the higher value grant is actually the fiftythou grant, not the hundred thousand dollar grant. Yeah, and so itallows you to create a more efficient work stack and to actually understand, likewhen it's worth it to like bring another people who you should be putting yourstaff on. And then the other question that allows you to ask yourself,which is really important, it's, Oh, this hundred thousand dollar grant is onlytwenty five percent. Is there something I can do? Is there informationthat I can uncover? Is there something I can do in terms of theway that I present the information or something I can find out about, forexample, what the award body with the foundation is really looking for in termsof the way that I could write this grant that's going to increase that percentagefrom twenty five percent to higher. Yeah, again, it's the non obvious insight. We're almost at the end of our time together, sadly, becauseI feel like we could talk for a long time. If you have acouple like methods for, you know, structures for helping people thinking bets?Let's not give them all only because we want people to read the book,but what are some touristics that people can...

...use if they want to embrace thisconcept of probabilistic outcomes and working on their decision making process? What are afew mechanisms that are useful? Well, yeah, thanks for asking me them. I so the first thing is to actually ask yourself, when you knowwhat I'd be willing to bet on this. So a lot of times we thinkthings are sure things and when you actually ask yourself, like well,what I bet on this, like what if someone came up to me andsaid, do you want to bet? How would I feel about this,you find out that it's not so much a sure thing. So what abet does it? It actually causes this uncertainty to bubble up where you startto view it in a more realistic way. It's a good way to get tosort of what your probability estimates are of outcomes is to think about themexplicitly as that, as if someone's challenging you to a bet. But themain actually thing that I would say, and there was some hints in it, and what I said in terms of like go talk to other people whohave different perspectives and make sure that the person that you're reporting to is partof the process and you're getting their perspective as well, is that you knowthe stuff that we do like, for example, the resulting bias or inmy book I talk about self serving bias, I talk about motivated reasoning. Thisis really part of the way that our brains are wired, and confirmationbias and hindsight bias and all of these things it's just very hard on ourown for us as individuals to be able to overcome. And it's not thecase that just because you happen to know about a bias and just because youhappen to be a smart person that somehow that's going to make it significantly better. It really doesn't, because, you know, you just kind of hadto accept like this is the way I'm wired, this is why everybody's wired. But the good news is that when we get away from ourselves and westart to look at other people, we can actually see bias and other peoplepretty clearly. I don't know if you've been watching any news lately, butlike when you're listening to the pundit speak, you're like they're biased, they're bias, this is what they're biased and you know, we can see that'svery clearly when people are sort of engaging in this kind of reasoning, whenit's somebody else. So we can actually take that to really create an intentionalway to solve these kinds of thinking so that we can get to a moreaccurate model of the world, which is to say, if I'm really goodat spotting other people's bias, they're probably really good at spotting my bias.So why don't I go find a few people who are willing to engage inthis process with me, who are really interested in whether they're accurate, whetherthey have an accurate model of the objective truth, whether they're moving toward thatas a goal, versus just the firm in the things that they are alreadybelieved, versus just sort of wanting to pat themselves on the back when theythink that they can take credit and cast blame where they don't want yeah.Right, if I can find some people who are like no, we're completelyin the accuracy business, and let's do that together and let's change the waythat people normally interact with each other, where we're going into this with acontract that not only are we going to be really open minded to those viewpointsthat disagree with us, but we're going to be grateful for it, becausethat is going to help with calibrate our beliefs, and that is what weare going to get our high from. We're not going to get our highfrom just like I knew my opinion was right, I'm so smart, butrather we're going to get our high for him saying, oh my gosh,like, thank you so much for giving me that perspective, because that hasactually changed my mind. Yeah, well, willingness to change your mind I thinkas a hallmark of, I don't know, something good intelligence. Ifyou're if you can't change your mind with one presented with new information, thenthen it's clearly not an objective opinion that you have right. And here's thething. I don't want to say that the goal is to change your mindevery single time that you hear an opinion that disagrees with you. I mean, obviously, sometimes the things that you write you believe are mostly true.But when you really listen with an open mind to opinions that disagree with you, either your mind will change or you will not only better understand your opinionbecause you will have had to defend it or against somebody who really truly believessomething different, and you will better understand why that person believes what they do. And that's actually really, really important, because let's say, for example,that you're competing for a sale with somebody else who has a different viewpointof what, strategically, what you're supposed to be doing in that situation,right, and so you have sort of competing strategies and you're executing strategy andthey're executing strategy. Be If you don't completely understand strategy, be number one, you can't sort of understand what about that might be good that you shouldincorporate into your own life. But also, even if you decide that it's nota particularly good strategy, really deeply understanding what they're doing and why they'redoing it is going to help you to more effectively execute against it. Yeah, actually beat it better, because you're actually taking the time to actually understandit, not in a straw man way,...

...in a really dismissive way of that'sjust dumb, but to actually really take a deep dive on understanding whythat person is doing what they're doing, and that's actually incredibly helpful to you. I completely agree. The last final thing I will say, because Ijust listen to this podcast and it was a different podcast about a different cognitivepsychologist actually who was talking about because I just think it's fascinating that I agreewith you that you sort of you want to get a tribe of people andsometimes your spouse is that person that helps you pursue and seek objective reality.Then that runs counter to some of the work that other cognitive psychologists have done, which says that the objective reality that humans perceive is it's like a userinterface. It's a construct for the world, but the world doesn't actually exist theway that we perceive it. It's a construct that's been developed that's evolutionarysuccessful. So I don't know. Anyway. So actually I completely agree, andI think that that's the point, is that if our goal is tobecause what we're doing in terms of our brains, as we're always modeling theworld, we have a model of the world. We're not seeing the worldas it is objectively true. We're interfacing with the world through this model thatwe have of it, to our experiences of it. So actually fully recognizingthat is what causes you to go and find a really good decision pod andsay we want to try to come up with the most accurate model of theobjective truth. We want to try to get to what's objectively true as muchas possible instead of just going around and believing that the things that are goingon are in our brain are right. Yeah, so that requires an openmindedness, because what that means is that if you just go around and youthink, well, what I'm seeing is the objective truth, then when someonesees the world differently than you, you just automatically think they're wrong because youdon't recognize that there's an interface between you and the world. Right, soyou just think, well, I just see the world for as it isand if you don't see it the same way I do, then I don'tknow what's wrong with you. You're blind or dumb or something. But onceyou recognize exactly what you said, which is that we're just operating off ofa model of the world, then what you recognize is that well, otherpeople have different models and my model is not a hundred percent correct. I'mjust trying to get it as close as possible. So I should always betrying to calibrate this and I should always be listening to other people who viewthe world differently than I do, with an open mind, because maybe theyhave something more right than I do. Yes, I agree with you.The only time I probably disagree is if you've done all of the work andall of the thinking and you have what you believe to be a you know, you've done as much work as you can, you have a model thatyou feel is accurate. There are still people that disagree. You've considered theiropinion and they're still wrong. I'm obviously speaking of politics, but you know, at that certain point, you know like there's I can't be both sideson everything. You know the world, the Skuy is blue. I haveto believe that. Yes, so you know what. So I think thatthat goes to strong convictions, Lucy Lee held. So when we're making ourdecisions, we want to make them in a strong fashion, saying I've donethe work, this is the best information I have right now, and sotherefore I can decide off of this, but always sort of be open mindedto listening to the other side. Now here's where I don't listen to theother side when I don't think that the other side is engaging in a genuineway, and in politics I think that that happens a lot, right.So I think that people are just engaging not for the purposes of seeking truth, but for the purposes of causing controversy or being provocative or you know,whatever it might be, and then then I'm not going to really consider whatyou're saying very much because I don't think that you're engaging in an authentic way. Right. So, as long as you authentically believe the thing that you'rebelieving and you authentically think that this is true and that this is what's bestfor the world, I will always engage with you. And even if I'vecome to the conclusion that I'm really sure that I'm a hundred percent right,I still think that I'm supposed to really try to understand your side because,first of all, it helps me have compassion for you and not think thatyou're a bad person just because you believe something differently than me, which Ithink is actually really important. The other thing is that it really helps meto sort of decide in relation to you right, like if we have competingvalues, I need to think about that. If I want to execute on thethings that I believe are right, I need to really understand genuinely whyyou disagree with me so and then again. And the other thing is that sometimesI might change my mind a little bit through that re engagement. SoI'm always sort of trying to do that. That being said, I don't wafflewhen I go vote right because at the moment that I vote, I'mvoting based on the opinions, in the beliefs that I have right now andwho I think is going to execute on those opinions and beliefs the best.And I do that with absolute conviction. And then after that I'll engage withsomeone who voted for somebody else to try to understand why. It makes alot of sense. Annie. So the book is called thinking in bets.What is the rest of the title? I don't have any murder decisions.When you don't, when you don't know all the fact, that's exactly so. Every decision, every do you have a preferred as there like a booksellerwhere you get a higher commission or something like that, or just happends onyou know, I don't know, like just go go to your local bookstoreat, you know, Amazon. I'm...

...a big supporter of Indie book soI love it when people support in the independent bookseller. So, you know, if you can do it that way, that would be amazing. There's actuallya cycled in new book. The best way to engage with me toa social media is on twitter at Annie Duke. I'm actually incredibly active.I create all my own content for that. And then another way to engage withme is I have a newsletter. If you go to my web site, Annie Dukecom, you'll see our chives of the newsletter, and the newsletteris actually basically just executing on what we talked about today. So taking events, current events that are either, you know, things in science or,for example, politics or technology or whatever it might be, in really applyingthis kind of framework to thinking about what's going on in the world. LikeI say, you can read before you buy, before you subscribe, butit's about two to three thousand words a week that you'll get for me,really kind of practical application of this kind of thinking. The other thing isthat there's a contact forum on there and if you write me, I willrespond. I don't think that I haven't responded to anybody, but I meanI try. I try to respond to a hundred percent of the people whowrite me because I love hearing from people who read my newsletter, who who'veread the book, because that's the best way for me to learn and tounderstand, you know, what people are thinking. And I've also gotten somereally cool pointers to other really cool sources of information from people who are readersof mine or different just different thoughts that they've had, and so that's areally fun thing. That's fantastic. So listen. Thank you so much forparticipating and for joining us today. The book one last time is called thinkingand bets, making smarter decisions when you don't have all the facts and hethanks for joining us. Thank you so much for having me. Take care. He folks, this is Sam's corner. Was Really honored and excited to haveany duke on the show. Any, as we know, has one multipleworld series of poker bracelets use on the celebrity apprentis. But most importantly, she's a trained cognitive psychologist and studied at the University of Pennsylvania so incrediblyinsightful, and her new book is called thinking in Bets and the whole pointof it is that we have to understand that decisions are bets on the future. I think that's a really interesting way of thinking about it. And Lifeis probabilistic and when you're resulting, which is the phrase that I think theyuse in poker, which is focusing inordinately on outcomes, you're not getting thewhole picture. When it comes to sales as specifically, comes down to forecastingaccuracy as an important determinant of the efficacy and success of our work process andof our sales process, and Annie mentions that under forecasting and over forecasting areboth equally bad. So we need to diagnose and dissect the process. Evenif, you know, we say the deal is going to come in fora hundred grand and it comes in for two hundred grand, yeah, that'sa great upside surprise, but we have to understand why we were inaccurate andwe also have to not in ordinarily focus on the outcome, because the outcomeis a result of a series of probabilities. We have to sort of understand statisticsand focus on the most probabilistic outcome, understanding that probability means that sometimes it'snot going to go our way. I think it's a really helpful frameworkand it helps keep us and keep our perspective in line as we go throughthe world of sales. I also think she talks about assembling a tribe ofpeople that are going to help you pursue objective reality, which is a reallya way of saying people that are going to challenge you on your assumptions,but in the spirit of pursuing truth, not in the spirit of, youknow, sort of personal denigration, and for a lot of us, likeme, my wife does that. But you know, get in a companyor a culture where honest, direct feedback, not about you as an individual butabout the process, helps you challenge your assumptions, change your mind andupdate your perspective. So that has been Sam's corner. It is perhaps amore cerebral episode, but one that I really enjoyed and I really think thebook is Great. Now to check out the show notes, see upcoming guestsand play more episodes from our incredible lineup of sales leaders, visit sales hackercomand head to the PODCAST TAB. You'll find us anywhere podcasts are found.If you enjoyed the episode, please share with your peers on Linkedin, twitteror elsewhere. Always unclear what that word in the copy means. I guessfacebook. Maybe write a handwritten note and send it in a letter. Ifyou want to get in touch with me, find me on twitter, say mF Jacobs, or on Linkedin at Linkedincomm F Jacobs. Once again,big shout out to our sponsors. Are Call, your advanced call center software, complete business phone and contact center, one hundred percent natively integrated into anyCRM and outreach a customer engagement platform that it helps effectively and efficiently engage prospectsto drive more pipeline and close more deals. Now I will see you next time.

In-Stream Audio Search

NEW

Search across all episodes within this podcast

Episodes (355)