The Sales Hacker Podcast
The Sales Hacker Podcast

Episode · 3 years ago

31. Annie Duke, Author of "Thinking in Bets" and Celebrity Poker Player

ABOUT THIS EPISODE

This week on the Sales Hacker podcast, we are delighted to speak with world-famous author and teacher, Annie Duke. Annie is a former World Series of Poker winner and has written an incredible book detailing how thinking in bets can improve outcomes. In this episode, we take those concepts and apply them to sales.

One, two, one, three, three, hey everybody, it is the sales hacker podcast. It's your host, Sam Jacobs. I am the founder of the New York revenue collective. I suppose I'm chairman of the global revenue collective. I can give myself any title I want when it comes to the revenue collective. I'm also the crow at a place called Behaveos, which is a big day of machine learning platform. And today we've got an incredible interview because we've got any duke, the world famous poker player. She won the world series of poker a few times. She was on the celebrity apprentice. So she has experienced with our wonderful president and she's also exceptionally insightful and she's written a book called thinking and bets and I encourage everybody to read it. We talked about that book in this episode. So before we get into the episode, let's thank our sponsors. We've got to the first is air call. It's a phone system designed for the modern sales team. They seamlessly integrate into your crm, they eliminate data entry for your reps and they provide you, if you as the manager, with greater visibility into your team's performance through advanced reporting. I when it's time to scale, you can add new lines and minutes and you can use incall coaching to reduce ramptime for you new reps. so that website is are called that I owe for its sales hacker. That's once again are called DOT IO. Forward sales hacker to see why Uber, done and Bradstreet, pipe, drive and thousands of others trust are call for their most critical sales conversations. Our second sponsor is outreach, dot Io, the leading sales engagement platform. Outreach triples the productivity of sales teams and empowers them to drive predictable and measurable revenue growth by prioritizing the right activities in scale and customer engagement. With intelligent automation, outreach makes customer facing teams more effective and improves visibility into what really drives results. Hop over to outreach that I oh forward sales hacker to see how thousands of customers, including cloud era, glass door, Pandora and Zillo, rely on outreach to deliver higher revenue. Her sales up. So, without further ado, let's listen to the interview with Annie Duke. High folks, it's Sam Jacobs, and welcome back to the sales hacker podcast. Were incredibly excited for our guest today, Annie Duke, who is an award winning author, Professional Speaker, decision strategist and also former professional poker player. Annie has just released a book, or I guess, earlier this year. The book is called thinking and bets, making smarter decisions when you don't have all the facts. I first found out about the book through mark and reason and his twitter feed and then I downloaded it, read it and was it just really, really intrigued by the ideas present in it, and so that's how we got Annie onto the show right now. Let me just tell you briefly about our background. For two decades and he was one of the top poker players in the world. In two thousand and four she won the world series of poker, her first bracelet. She also won a big tournament that year in the Tournament of champions. In Two thousand and ten she won the prestigious NBC National Heads at Poker Championship. She's also, by the way, a student of cognitive psychology, so she was also awarded a National Science Foundation and fellowship to study cognitive psychology. At you pen. She now spends her time writing, coaching speaking on a wide variety of topics. She's regularly sought after as a public speaker. She is a philanthropist. She's written other books about poker mostly, and this is sort of I think, but she'll correct me if I'm wrong, one of the first books that she has it's really not about poker at all but just about decisionmaking. She's a master storyteller. It works. Incredibly excited to have her here on the show. So welcome Annie to the show. Thanks for having me, Sam. We're excited about it. So the context for this conversation is I bought your book. I read it. I thought it was really, really interesting and I think it's incredibly relevant to the sales profession. But first let's just hear a little bit about your background again. We know you, I think, is now a professional speaker and decision stratagist, as you say, and a coach, but I think a lot of the world knows you as a poker player. Where did you come from? You know where to grow up. You talk about this in some of the book, but walk us through how we got here a little bit. We're okay. Well, I was bored in a small campsure. Well, I mean I'll start with my adult life. I started off doing my Undergrad work at Columbia. I went into major actually an English lit and but ended up double majoring in English and psychology, partly because of the mentorship of wonderful professor. They are named Barbara Landau and I was actually her research assistant for four years and she was working in the Psyche Department there and she encouraged me it took on to Bradwood School, which I did. I got into the University of Pennsylvania Program and got, as you said, in National Science Foundation Fellowship to study cognitive science and I was working with Lylon Henry Gliteman, who actually the book that I wrote is dedicated to they're really my intellectual parents and most ways, and so I was studying there and I was there for five years and did all the things you're supposed to do to, you know, get your PhD. I studying in specific first language acquisition. But what kind of psychology is really dealing with is how do you interpret the world around you? Right? So...

...they're dealing with questions that kind of cross with a lot of different fields and disciplines. You know, biology, physics, physiology, a little bit of philosophy, business, psychology and it's some computer science, and that's really put together into this kind of cross disciplinary view of the way that we interpret the world. So a lot of what I was thinking about in particular was learning and how you learn, and in my particular interest was and how you learn a first language. So I'm going along and right at the end, so this is after five years, I'm going off to interview to become a professor for my job talks and academics, and I had been struggling that year with some stomach issues that got really, really bad right around the time that I was supposed to be going out for my job interviews and actually on my way to Nyu for my first job talk, I got so sick that I landed in the hospital and I was there for actually two weeks, and so I had to delay going out onto the job market, and in academics that market is seasonal, so I couldn't just delay it a few months while I got better. I had to delay it a whole year. So during that year when I was recuperating, I wasn't teaching, which was one of my sources of income. I certainly didn't have a fellowship. So I just found myself in need of money, honestly, and my brother, Howard letter, had already started playing professional poker at that time, through somewhat circuitous route, if you we can talk about that if you want, but he had already found his way to poker, which is very unusual at that time because poker was not on television. It wasn't something that as a kid you could just sort of like see, oh, this is something that I want to do. It wasn't really a known profession, but he had found his way there and so I was aware of it and he suggested to me, Oh, since you need money, this would be a good thing to do in the meantime and I'll teach you how to play, like you've watched me play some and I'll teach you how to play. So it was during that recuperation time that I started playing and I thought, okay, I'll do this and then next year I'll go back out on the academic job market. But I ended up really taking to the game. I really, really loved it. I love the problem that it was presenting to me and you know, in retrospect, I now realized that I didn't actually change what my area of interest was, because when I was playing poker, what I was really trying to figure out was, you know, i. How are you learning this game where you getting tripped up? How do you deal with this very noisy feedback that you're getting from the way that things are turning out? How are you mapping out the future when it's probabilistic? You know, all of these kinds of questions that were actually very similar to the kinds of questions that I was asking the graduate students. So essentially, the meantime turned into like twenty years, eighteen years or twenty years, and then I retired in two thousand and twelve. And this was all in Montana. That's I mean, I think, an interesting wrinkle right, if I'm not missed. So it started in Montana. So I was I was obviously in Philadelphia when I was at you pan, and when I was recuperating, my then husband actually had a house in Montana. His family lived in Montana, so we just decided we would go out there. Well, I was trying to get better instead of staying in Philly in my apartment there and, you know, me continuing to do research that it would be better if I was really trying to take some time off. So that's how I ended up in Montana and I was there for about three years. That was really where I would say my apprenticeship occurred, and by the time I was done there I had declared myself professional. I had already cash in three different world series of poker tournaments by that time and I'd actually made a final table and come in second one. And so at that point I moved down. I'd made the decision to move down to Las Vegas and pursue it full time. So that that was in one thousand nine hundred and ninety four that I decided to pursue it's full time and become a professional. Then the real kind of Aha moment for me, I think, was in two thousand and two. So now this is eight years sin too my professional life as a poker player. A friend of mine named Eric Seidel, who has mentioned quite heavily in the book as you know, who really I consider one of my main mentors in the game. He had started his life off partly as an options trader and he'd been trading on the floor in New York and also I think in La and he had made. He had developed obviously some relationships with that and one of those relationships that guy named Roger Lowe, had asked him if he would come and speak to retreat of options traders. So Roger Lowe had a hedge fund at that time and had a group of options traders and he wanted Eric to come and talk to them about how poker might inform of your decisions about trading and risk. So Eric is notoriously shy about public speaking. He really does not like getting up in front of large audiences and talking. so He'd demurred, but he said well, maybe you should have anny do it, because Annie actually used to he shoes to teach...

...college. So I assume she's pretty good in front of an audience since she has a lot of practice at it. So I got asked to do it and it was it was really actually the first time that I thought in a very explicit way about what the relationship between my academic life was and what I've been doing it at poker and really started to explicitly figure out how can I express this relationship between these two things and what poker might teach you about the challenges of learning and how we're thinking about the future, how we control our emotions when we're in the moment of a decision, what kind of biases are really tripping us up and preventing us from really experiencing the learning curve that should be available to and poker because the amount of feedback that you're getting. So it really brought to light at this this really big problem that I sort of realized that I'd been trying to tackle, that I've been thinking about a lot as I was learning to play the game, which is that in graduate school, you know what I had been taught with that learning occurs when you have lots and lots of feedback tied closely in time to decisions and actions. It's very common thing. You hear it in every single psych one class and I thought, well, this is interesting, because here I've come into this game. You know, when I was playing in Montana and I was playing against people who have been playing the game for decades, and you know, certainly I did. I did have a very good teacher and my brother at that time, but certainly did not have the experience. I did not have the number of hands out of my belt that these guys had, and honestly, they should have been a lot better than I was, you know, just given the sheer amount of experience that they had had. But yet they weren't. And you know, I didn't think it was something particularly special about me. I thought it was something special about the problem that poker was presenting in terms of how you actually learn from what's happening around you. And what I realized was that this idea of, you know, learning occurs when you have lots and lots of outcomes tied closely in time to decisions and actions, is missing a sentence, which is except when the feedback is really, really noisy, and what that means is when the actual outcomes that you're getting are only very, very loosely related to the decisions and actions, which is exactly what's happening in poker. So over the long run in poker, if I'm playing really well against you, I'll end up winning over the long run. Likewise, if I'm playing worse than you, over the long run, you know that difference is going to reveal itself, but it takes a while. So on any given hand, or even on any given, you know, eight hour session that I might play, I could end up winning even though I've played quite well in comparison to you, or I could end up losing. I could end up winning even though I quit played quite quite poorly, rather in comparison to you, and I could end up losing even though I bet I've actually played quite well in comparison to you. So this present a really big problem because whether you've played the hand well or not is kind of hidden from you because I can't see the other players cards, so I don't really know what they're holding. And even if I could see the other players cards, I still have to guess about how you might react to them. So I have this sort of two stage process, which is guessing at what your whole cards are, which are faced down to me, and then also trying to figure out how you might react to those cards. Are Certain Bets, and at the end of the hand that stuff is mostly never revealed. So I'm just taking the outcome I want or lost the hand, and I'm trying to work back to uncover this thing that's actually pretty opaque, and so it's very noisy because it's not a perfect signal and the way that you can sort of feel like, okay, it's not a perfect signal is to think about it in comparison to a game like chess. So in chess, if I lose to you, like, if I play a game of chess against you and I lose, like, what do we know about my decision making in comparison to your? Yeah, well, we obviously know it's a very discrete outcome. So we can say it was either worse or better. And and I think the other point is that there's always a I mean the maths would say that there's always a correct decision in chess, correct and there's probably not in poker. I would guess. Well, there's a theoretically correct decision in poker if you could turn poke into a game like chess. So you know, we could think about that comparison. Right. So in Che the reason why, in chest there's a theoretically correct answer that we can go back and work out, and the reason why we know that, if we if I have a bad outcome, that I can work backwards and say that I've made worst decisions than you, is because you're losing a lot of this noise, right, a lot of the uncertainty, a lot of this sort of probabilistic nature of things when you're playing chess and you're losing it for two reasons. One is that there isn't a super strong luck element, meaning somebody isn't randomly coming across and like taking, you know, a pawn off your board or something at like random intervals where you don't know when that's going to happen, which is a little bit what happens at poker when and there's a bad turn of the card. But also there isn't a hidden information element, so there's...

...no information asymmetry. I can see your position just as well as you can see my position. Yeah, and what that means is that we can actually work it out. So as a player, I can say, like I can see what all of your possible moves are and I also know that the the pieces are only going to move on purpose, right, because someone does something on purpose to make the move that way. So it just presents like a super different problem that takes the noise out of the problem in a way where you can actually work backwards and try to figure out, you know, why you want are lost, and what that means is that learning proceeds at a much more orderly path in a game like chess. Now chess is very complex. So it's hard, you have to it's a lot to master and most people, obviously, you know, aren't going to master it. But the learning proceeds more orderly and generally. You know, someone who had been playing chess really seriously for twenty years would always crush someone who came in essentially on the first day, and that's not necessarily the case in poker. So poker really presents a unique problem and the thing that I think I found really fascinating was it's the unique problem that I was studying when I was in graduate school, because poker looks a lot more like the decisions that we make in life. tegging it back both to first to the audience, which is sales, and sales by definition is probabilistic, you know, and our win rates from this maybe to inside baseball on some of the jargon, but you know, we create opportunities that sort of like the notion of a contain sales conversation. Those opportunities should win on the low end, depending on how you define it, maybe like eight to ten percent of the time and then on the very high end, from two thousand and twenty five thirty percent of the time. So either way we're losing most of the time. Yep, and similarly, I mean similarly in Poker, I'm sure you you lose more hands, or at least you fold more hands, than you win, and you have to know how to size the bet and do all of those things. But one of the key concepts in the book that you articulate is this concept of resulting. So walk us through you know like what that means, because I think it's and we were talking about it, you know, offline before, but it's probably like the most prevalent thing that happens, you know, certainly in sports, but really in life, which is so tell us what the definition is? Yeah, sure. So resulting is essentially working backwards from the quality of the outcome to the quality of the decision. So it's basically saying, if I can see what the quality of the outcome is, then that tells me what I need to know about the quality of the decision. Now, resulting in something that's particularly a problem when you're trying to evaluate what somebody else is done who you aren't in direct competition with. So this is why we see it so much in sport. So the example that I open the book with obviously is Peak Carol, the Seahawks, two thousand and fifteen super bowl forty nine, last play of the game, and so they're against the New England patriots at second down. They're down by four, so they need to score a touchdown here and they're on the Patriots one yard line and they have one time out left. So this is like one of the most amazing examples of resulting that that I've ever seen in my whole life, which is why I open the book with it. So basically what happens is that Pete Carroll is expected. There's an expected play here, and they expected play is that pee carrol's going to hand it off to their running back, marsh on Lynch. Well, he's moll is it personally going to hand it off? He's gonna have Russell Wilson handed up. That's right. Yeah, and and Marshawn Lynch is one of the best short yardage running backs in the history of the game, which is I need. Actually he had had a great game and, you know, run through a bunch of Patriots earlier. So right, exactly. So everybody thinks, okay, he's going to hand it off to Marshall Lynch, that's the obvious play. Marshall Lynch is going to try to get the one yard. You know, obviously if Marshall Lynch fails there, they have this time out in their back pockets, so they'll call the time out and they'll take another shot at it, and that that's the way that they think that the game is perceived going to proceed. But Pie Carroll actually does something super surprising here. He has Russell Wilson actually passed the ball. Russell Wilson does that. He passes the ball and it's pretty famously intercepted by Malcolm Butler and obviously that ends the game and I really highly recommend that people go and try to find the footage on Youtube because I want you to listen to Chris Collingsworth call this and you can hear Chris Collingsworth just just you know, this is the worst place ever seen. Basically, like I don't know what he was thinking. This is so ridiculous. Why did he make this call and then the next way, even after people had had time to think about it? So, you know, in Chris Collins Worth Defense he's making this judgment obviously on the fly right then. But the people who are writing it about it the next day. I have had a lot of time to think about whether this play was any good or not. But they didn't really say anything different than Chris collings worth said, and there seemed to be kind of a the only disagreement among the major newspapers was was it the worst play in Super Bowl history or was it the worst play in NFL history period? Now when I think is really interesting about this is that what we do know is that it was definitely one of the worst outcomes in Super Bowl history,...

...that's for sure, and for sure it was one of the worst outcomes in NFL history, although I would argue like that Joe, thisman getting his leg broken thing, which that was probably worst outcome, at least for Joeth Siseman, but and for Washington Redskins fans, of which I am one. Yes, exactly, but but you know, it's certainly it's going to be competitive here for one of the worst outcomes in Super Bowl history. But that does not mean that it was such a terrible decision as everybody said. And there were a couple of voices that were arguing a different stance. One of them was Benjamin Morris on thirty eight. Another one was, I think, Brian Butler on slate. You can go read them if you want, but basically the point that they were making was whoa hold on a second guy's don't you want to kind of know what the interception rate is here, because if it's super low, maybe we should start to think about what was Pete Carroll getting in exchange for the play? So if we start they're a super conservative estimate of how often that ball is going to be intercepted. There is around two percent. The range of estimates is going to be between one and two percent. So we already know that this was a really, really low probability event. So let's just start there. And then the second really key piece of information is that if you hand it off to Marsha on Lynch, remember they only have twenty six seconds left. The only way to stop the clock if Marsha on Lynch fails is to call the time out, and by the time that that has occurred a lot of time will have rolled off the clock because obviously a run play takes longer than a pass play and by that time they're only going to have a chance, one more chance at the end zone. But if they pass the ball there's really kind of three outcomes that occur. We're setting aside fumbles, which we should, because you can fumble also if you're running the ball, and the three outcomes are an interception, which is what happened, which we know is only between one and two percent of the time, an incomplete pass, which stops the clock very quickly Aha and allows you to run the ball twice again, or a touchdown, which obviously wins the game. So that's that key piece is that when the ball is incomplete, the clock stops so quickly that Seattle can still hand it off to marshawn Lynch twice. So basically what they were arguing was for this little like one to two percent chance that this disaster happens, what you're getting in exchange is three tries at the end zone instead of two. You still get those two running plays. If you happen to fail on the past side, almost all the time that you fail, you still get the too run play. So you know, you can agree or disagree with that. Like I'm willing to have a conversation with you about, you know, where the stakes too high or, you know, was that too high of an interception rate to Willie with and but I think that we can get to really understanding what's going on here, that they were really over lying on the fact that the outcome was so bad. But just asking a simple thought experiment, and the thin simple thought experiment is, what do you think would have happened if the ball was caught for a touchdown? What do you think Chris Collingsworth would have been saying in game and what do you think that the headlines would have read the next day? Now, brilliant call, of course, yes, and actually I mean, just to sort of prove the point, you can go listen to Chris Collingsworth call the PHILLY special from this year Super Bowl, which was an incredibly unexpected play. The Eagles were on the Patriots one yard line again and it was fourth down and the eagles were up by two at the end of the second by three rather at the end of the second quarter, and everybody expected they were going to go or field goal and go into the locker room up by six. And instead, not only did Doug Peterson not put out the kicking team, but instead he went for it and Nick foles went around and caught the ball in the end zone somehow. So the quarterback ended up in the end zone as a receiver. Now, that play happened to work out equally unexpected and you know, Chris Collingsworth was calling him a genius and game in the next stay. Of course, that was all the talk of the newspapers about how smart Doug Peterson was. So we actually have an example of what happens when you have two kind of equally weird plays and in one case it works out and in one case it doesn't. And I think that we can also do the reverse dot experiment on the Philly special and say, well, what if that ball had been intercepted, they had not gotten the three points, you know, in the Eagles had gone on to lose the game, no doubt people would have been going and pointing at that particular play. So what we can learn from this is that we really make this really big mistake, which is we think, you know, if we have a good outcome, if we close the sale, and we obviously had a brilliant process and we should try to repeat that, we should be, you know, patting ourselves on the back for how great a job we did. And if somebody, if we see somebody not close a sale, if someone's working for us or whatever it might be, then we're much more likely to sort of point fingers and try to dig in and look at where the mistakes were and where the blame was. But what we know is that things that turn out badly aren't necessarily just because of mistakes, and things are their turn out well aren't necessarily a mistake free yeah, I mean, I think exactly...

...to your point. The last thing on the football part is that I think the the statisticians have done the math and it. I think the result is that, like in almost every situation, very rare, I say the majority of situations, people should go for it on fourth down and they never hurtetting, by the way, on your own one yard life. Yeah, exactly, but they never do right. So, yeah, and actually, so the reason why I should go for it on your own one yard line, just for people who are a little bit more statistics x Geeks, is because when you punt, the other team almost always ends up inside the fifty and so they're already kind of guaranteed three points anyway. Yeah. No, so you're already giving up three points and so you're just supposed to try to go for it on and the one yard line and on your own one yard line on fourth down and just try to get the first down in order to move the ball a little bit farther down the field. The fascinating thing is the the sports journalism world, in the media, because they are so heavily focused on outcomes, they'll construct entire narratives out of, you know, the Grit and the virtue and the determination. Those are all adjectives that are going to be used on a probabilistic outcome where you win and then if you again make the same decision and you lose, then it's something about your spirit and you're no, you're not strong enough and there's weakness and the coach wasn't good. So that's exactly that's exactly right. That what's happening is that on the downside, people are looking for things that were in control of the team, right in the control of the coach, that, you know, they just didn't have the spirit to win or, you know, they weren't good enough to overcome or, you know, whatever it might be. And also on the upside, they're also saying, oh, that was because of their grit, but we know there's a lot of randomness right, like if you think about basketball, for example, when you're talking about like a one point lots or one point win, a lot of it just has to do with sort of randomly, you know who ends up with the ball last. Yeah, I mean, I mean I'm a hockey fan, so there's nobody that knows about probabilities more than a questioning capitals hockey fans. And, by the way, so speaking of the fourth down thing, the NHL is another lead that really has this problem, because there's been a lot of math done on when you should pull the goalie and people should be pulling the goalie a lot earlier in the game than they actually do. And why don't they do that? Well, because they know that the fans and the ownership are all results. I mean this is a problem, right, like Pete Carroll got so excoriated when this didn't work out well, and that happens to NHL coaches as well. If they pull the goalie in a place where people don't kind of sort of accept that that's maybe something that you should do, then if they go on to lose the game, what's going to happen? Like people are going to point at the coach and say, I can't believe how bad they are. Oh, clearly, you know. Clearly there's such a failure. And here's the problem. So this is where we can really kind of tie these together and see how this really affects behavior. Is that now people see the treatment that Pete Carroll gets. People see how, if you pull the goalie, you get completely yelled at. You know, people see what happens when you go for it on fourth down and under unexpected situations and you fail, what kind of treatment the press gives you and the fans give you. So what does that drive people to do? It tries people to not make those decisions. So even though the math is very, very clear on these kinds of things, the NFL has been incredibly slow to adopt these fourth down plays and the NHL has certainly been incredibly slow to adopt pulling the goalie relatively or early in the game. So why is that? Well, because they're afraid of basically getting yelled at. So we can sort of go back to this in terms of what's happening, for example, in your world in terms of sales. So you have this world where your prospecting and some small percentage of those prospects are going to actually realize, and the idea is that the ones that do realize are going to pay you enough in order to sort of compensate you for all the small fishing lines that you're putting out there. Right. But if I know that the person that I'm reporting to is going to result on me, which is what most leaders do, right, most leadership results, and if they didn't close the sale, they say, why didn't you close the sale, like, what did you do wrong? And I know that that's going to happen, then what is that going to drive me to do? It's going to drive me to try to close the sale under kind of under all circumstances. So what I'm not going to do is tend to push the boundaries. In other words, it's going to drive me to try to close as many sales as possible so that I can sort of fend off this resulting problem. I can spend off this kind of like when I don't close, I'm going to have this deep dive come in on me. Right. So it's going to cause me to maximize the number, not maximize the amount, and that's obviously really bad if people on your team are behaving in that way and if people on your team are behaving in that way where they're trying to maximize number as opposed to maximize actually how much they close the sale for. Generally, if people are behaving that way, it's your fault because you're...

...acting like the fans of the seahawks. Yeah, I mean then there's a again, whether it's sales or any other aspect of life, people fundamentally misunderstand like the notion of expected value. Right, I have a question for you in my world. So my world is enterprise sales, and these are twelvemonth, sometimes eighteen month sales cycles and one of the things that you mentioned poker so useful for is because you're getting feedback all the time on decisions that you make that are determining outcome. So you have enough data to analyze your process and to make improvements. How do you deal with the situation where these are binary outcomes? You know, we either close wheel or we don't, and for very large amounts of money. So they expected value is fantastic, but the forecasting of these deals is very difficult and also there's this resulting problem where I'm not getting enough. I don't have that much data for me to in any constrained a period of time, particularly it startups, because, you know, we're trying to like double and triple and twelve month periods, and so how do you deal with the situation where you're not getting enough feedback, there's not enough signal coming back to you in order to, you know, adjust and improve the process? Yeah, so that's a really interesting question. So let me just roll back sort of a little bit earlier and what you said to say it in some ways that slower cycle is actually advantageous. So let me just explain why. So one of the problems of poker is that you're getting so much feedback that the past that you're on becomes very, very clear to you. In other words, you know, you're sort of getting pounded by feedback and people aren't natural data aggregator. So it's not like I'm getting a series of results and I'm saying we'll let me wait and step back and sort of take the average of these results and kind of way to make a decision about what each of these outcomes means. And I'm going to step back and I'm going to wait until the end of the session to actually go back and try to figure this out. Instead, what we tend to do is we feel outcomes in sequence. So we feel them one at a time and the decisions that we've made about each outcome and affect the kinds of ways that we think about the next outcome. And only that you can see really clearly when you're sort of on a downward trend versus and the upward trend, and the way that we actually view what's going on around us is very path dependent. So the way that we feel about it, the way that we think about it, the way that we analyze it is very path dependent. So you're right, there's a lot of there's quite a bit of upside to poker in terms of the shortness of the cycle. You get feedback off your decisions within, you know, thirty seconds, and so that's really great to be having such a short cycle and it's really great to be having so much feedback come and such a short period of time. But there's also a dark side to that, and that dark side actually can cause people to get so sort of emotionally hot and be unable to even step back and aggregate those results in any kind of way that they can actually learn from it. It's part of the reason why somebody could be playing for twenty years and continue to make the same mistakes over and over again, partly because of the amount of data. So there's kind of good and bad it. So when you have a slower sales cycle, obviously there are challenges that come with that just in terms of your kind of fishing around in a space where you're not closing the gap that quickly, and so it feels like you don't have as much certainly it's true that you don't have as much data to work with in order to make your guesses and it's a slower process, but you do get a little bit out of that path dependency. It does allow you to approach it in a little bit more of a rational way. So I just want to just point out that there are some good things to kind of slowing it down. Yeah, yeah, we're going to say, oh no, I go ahead because I was just going to move on to the next point. So go ahead. The next point is probably the interesting one. Go for it. So, but obviously there are downsides to not having as much data as well, because you know now, as you're trying to make your estimates of what the probability is that something's going to close, you're sort of casting about a little bit more and your confidence intervals are going to be much wider around those estimates. And then very often the answer itself that comes in it's very remote from a lot of the initial decisions that you made, which actually increases the amount of luck in the process, because you know, you make a decision to pursue a particular lead and try to close a particular deal and it might be eighteen months before you actually get to the results. In a lot of stuff can happen in eighteen months that you literally have no control over. Not only that, within that eighteen month period there can be a lot of information that reveals itself that you didn't have access to at the time of your initial decision, and so that can make it hard to sort of sort out what should I have known or what could I have known, you know, versus what was reasonable for me to know what was only going to reveal itself after the fact, after I'd already started down the path. So that can become really difficult. Yeah, I mean, I think the one of the challenges is that the time interval that we're all being judged on, especially in sort of high growth startup land with these sales cycles is out of sync with the length of the sale cycle, because I'm I'm judged on a time interval that doesn't give me enough data to really build an efficient decision making process around these types of deals. Right. Yeah, so basically what I would say about that is...

...that try to look to the good in this, which is this that one of the ways that I recommend that people deal with this outcome by us, with resulting is to actually, when you're deconstructing the decision as much as possible, try to deconstruct the decision process prior to the outcome of ever having occurred. And if you can't do that just because of time constraints, then you should deconstruct the decision making process with people who you don't tell the outcome too. So in that sense it's actually kind of good news that your sale cycle is so long. But you have to get buy in from the people around you. In other words, you have to do a lot of really good work to work through what strategy you're going to use and make these reasoned estimates as to what the expected value is of that particular opportunity. And once you've made that reasoned estimate of what that expected value is. Then everybody has to buy into it and understand. And the reason why it's really important to actually make these expected value estimates explicitly is because in making those expected outcome estimate, you have to actually go through what the set of possible outcomes are. Right. So if this could in this case where it sounds like it's binary, it's either yes or no, right, it's either Yes for this amount or no for this amount. Yeah, I mean to not to interrupt you, but even just to underscore your point, it's actually not quite binary because it's yes or no, but it's really oftentimes it's no, which means not right now. It's only binary against a specific time constraint. It's not generally binary. Right. So let's talk about it, but as a specific time constraint, just to simplify it, because obviously we can do an expected value calculation that includes that has a conditional probability, which is no. But what's the probability that it will occur at a later date? Yeah, right. So let's take the simpler problem of it's yes or no and in any case it's always for the same amount. Right. So, in order to come up with an expected value. There you have to really think about what's the probability of yes and what's the probability of no, and you have to do that in an explicit way. And if you actually work through that process with a good team, where you're coming in with your own estimate, hopefully you're not telling them what your estimate is, as you're trying to get at their opinion about what their estimate on that is, because all of you have different experiences and different ideas and you're all the best experts in estimating the thing that you're trying to estimate here. Right and there isn't a right answer, because it is probabilistic in nature and what you're trying to come up with is your best guess. And once you've bought into that in advance, what tends to happen, because you can memorialize that and say, okay, for Opportunity A, here's how often we thought it would be as, how often we thought it would be no, and here's the expected value on it. For Opportunity to be the same thing for opportunity see for Opportunity d. Now there's all sorts of really good things to come out of that thing. Number one is because you've actually worked through that and gotten by and on it, hopefully, because we've worked through that as a grouper, or you've actually worked through that with the person that is judging you, so they can see what you're thinking processes and that they've made any tweaks that they think should go into that right. So they've sort of been wrapped into part of the process and so they have a little bit of ownership over it as well. Now, if it's yes, hopefully you're not padding yourself on the back too much because it was in the estimate already of yes, and if it's no, hopefully you're not getting judged too much for it. That's number one. Number two is because you're actually actively trying to make these estimates. It starts to get you to be information hungry. So one of the problems that we have, if we go back to that initial description that I said of poker, is that we've got two sources of uncertainty. One is hidden information, there's just stuff that we don't know, and the other is luck. So in order to make a really good estimate of how often you think it's going to be yes and how often you think it's going to be. No, you have to actually really try to uncover those two things. So first of all, you have to think really clearly about how much luck might be in the process, like, even if I do everything exactly right, right, even if I don't make a single mistake, even if I have all the information I need in order to come up with a perfect strategy and deploy perfect tactics and trying to execute on this, how often is this just out of my control and I have no say in the matter? So it gets you to think about that really clearly in advance. But the more important thing that it does is that, in order to come up with a really good estimate what you think those probabilities are in the future, you must become hungry information from information, because you're trying to get those to be more and more precise, because you're trying to model those outcomes as accurately as possible. In order to do that, you're trying to uncover really good inputs into that model. So you're trying to figure out, like, what information am I not thinking about? What am I missing? Here's a really important one. Why might I be wrong? which is not something that we tend to ask ourselves if we're not thinking about these...

...explicitly, if we're thinking supplicitly, we tend to ask why are we right? When we're actually trying to create an accurate model of what's going to happen in the future, we start to ask ourselves, why are we wrong? You're going to start to seek out opinions and you're going to be especially appreciative of dissenting opinions that actually help you to calibrate what your estimates might be. So get you to really focus on the uncertainty, because what you're trying to do is narrow down your own kind of cone of uncertainty. Right. You're trying to make that on error and it gets you to start to uncover this stuff. Yeah, and then the last thing it does is kind of the other after the fact of fact that it has that's really, really good is not only are you less likely to get resulted on, both resulting on yourself or having someone result on you because you bought them into the process, but now, when you go back and you're trying to get feedback from the leads that you were working with, instead of just going and asking the people who said no why didn't can you just give me some feedback as to why I didn't get it, which is what our tendency is. You're also going to go talk to people who said yes and say, Hey, can you just tell me, like, what is it about what I did that caused you to say yes? What actually do you think I've made a mistake on? What could I have done better? What do you really think the probability of yes was where you like always a yes and no matter what I did, I was going to get a yes. was there something in particular I did that caused you to say yes, even though I did that? Were you on the fence or was it a wide margin? And those are the kinds of questions that we tend not to ask when we actually get a yes, we tend to just sort of leave it and say, Yay me, I'm so excited, like look at me, I'm so excellent. But the problem is that it is actually really important for your ability to estimate properly in the future to know was it a narrow yes or was it a wide yes? Was it something you did or was it yes that you were right place, right time, like, and the reason why we don't like to go in and sort of Dick take a deep dive on this and explore it. Is because one of the things that we really like to do is get those pad on the back of moments, right, to get those moments of feeling a lot of credit. And if we go in in the person tells us like, actually, it was really close and I just flipped a coin, it feels like we're sort of losing credit for own part in whether that closed or not, because it feels like bad news then that it wasn't wide and it wasn't like I was so brilliant that it wasn't even close. But the thing is that when you actually refocus on my whole goal is always to be the most accurate estimator, like that's what I really care about, then that then becomes your reward and the fact that you're even willing to go in and talk to somebody and get some bad news on something that you thought was good news becomes what becomes sort of the self reinforcing action. And the other thing is that when we actually try to think it expected values this way. The other thing is that we actually get much, much better, it's at creating a good work stack right, like we understand what the higher priorities are. So it's an example. Like I was working with a nonprofit that was, you know, obviously has a lot of grant prospecting happening, and so it's a similar situation to you. It's very long cycle and it's either a yes or no. Right. So there's some award amount on the grant, it's a hundred thousand dollar grant. They're either going to get a yes or no and it's a long cycle decision. So what was happening when I came into consult with them is that they were naturally stacking the highest dollar amount, like the highest award amount grants, at the top, and that was the thing that they were spending the most time on. It was certainly what they were putting the most senior people on. If they were hiring an outside grant writer, you know, a contractor to come in to work with them, they were putting in them on these tie dollar account and when I had them come in and start doing this expected value work, what ended up happening was a much more efficient stacking because if you have a hundred thousand dollar potential award that you're going to get twenty five percent of the time that is actually the expected value on that is twenty five thousand dollars. But if you have a fifty thou grant that you're going to get seventy percent of the time, that's worth thirty five thousand dollars. So what happens is that you start to see that the higher value grant is actually the fifty thou grant, not the hundred thousand dollar grant. Yeah, and so it allows you to create a more efficient work stack and to actually understand, like when it's worth it to like bring another people who you should be putting your staff on. And then the other question that allows you to ask yourself, which is really important, it's, Oh, this hundred thousand dollar grant is only twenty five percent. Is there something I can do? Is there information that I can uncover? Is there something I can do in terms of the way that I present the information or something I can find out about, for example, what the award body with the foundation is really looking for in terms of the way that I could write this grant that's going to increase that percentage from twenty five percent to higher. Yeah, again, it's the non obvious insight. We're almost at the end of our time together, sadly, because I feel like we could talk for a long time. If you have a couple like methods for, you know, structures for helping people thinking bets? Let's not give them all only because we want people to read the book, but what are some touristics that people can...

...use if they want to embrace this concept of probabilistic outcomes and working on their decision making process? What are a few mechanisms that are useful? Well, yeah, thanks for asking me them. I so the first thing is to actually ask yourself, when you know what I'd be willing to bet on this. So a lot of times we think things are sure things and when you actually ask yourself, like well, what I bet on this, like what if someone came up to me and said, do you want to bet? How would I feel about this, you find out that it's not so much a sure thing. So what a bet does it? It actually causes this uncertainty to bubble up where you start to view it in a more realistic way. It's a good way to get to sort of what your probability estimates are of outcomes is to think about them explicitly as that, as if someone's challenging you to a bet. But the main actually thing that I would say, and there was some hints in it, and what I said in terms of like go talk to other people who have different perspectives and make sure that the person that you're reporting to is part of the process and you're getting their perspective as well, is that you know the stuff that we do like, for example, the resulting bias or in my book I talk about self serving bias, I talk about motivated reasoning. This is really part of the way that our brains are wired, and confirmation bias and hindsight bias and all of these things it's just very hard on our own for us as individuals to be able to overcome. And it's not the case that just because you happen to know about a bias and just because you happen to be a smart person that somehow that's going to make it significantly better. It really doesn't, because, you know, you just kind of had to accept like this is the way I'm wired, this is why everybody's wired. But the good news is that when we get away from ourselves and we start to look at other people, we can actually see bias and other people pretty clearly. I don't know if you've been watching any news lately, but like when you're listening to the pundit speak, you're like they're biased, they're bias, this is what they're biased and you know, we can see that's very clearly when people are sort of engaging in this kind of reasoning, when it's somebody else. So we can actually take that to really create an intentional way to solve these kinds of thinking so that we can get to a more accurate model of the world, which is to say, if I'm really good at spotting other people's bias, they're probably really good at spotting my bias. So why don't I go find a few people who are willing to engage in this process with me, who are really interested in whether they're accurate, whether they have an accurate model of the objective truth, whether they're moving toward that as a goal, versus just the firm in the things that they are already believed, versus just sort of wanting to pat themselves on the back when they think that they can take credit and cast blame where they don't want yeah. Right, if I can find some people who are like no, we're completely in the accuracy business, and let's do that together and let's change the way that people normally interact with each other, where we're going into this with a contract that not only are we going to be really open minded to those viewpoints that disagree with us, but we're going to be grateful for it, because that is going to help with calibrate our beliefs, and that is what we are going to get our high from. We're not going to get our high from just like I knew my opinion was right, I'm so smart, but rather we're going to get our high for him saying, oh my gosh, like, thank you so much for giving me that perspective, because that has actually changed my mind. Yeah, well, willingness to change your mind I think as a hallmark of, I don't know, something good intelligence. If you're if you can't change your mind with one presented with new information, then then it's clearly not an objective opinion that you have right. And here's the thing. I don't want to say that the goal is to change your mind every single time that you hear an opinion that disagrees with you. I mean, obviously, sometimes the things that you write you believe are mostly true. But when you really listen with an open mind to opinions that disagree with you, either your mind will change or you will not only better understand your opinion because you will have had to defend it or against somebody who really truly believes something different, and you will better understand why that person believes what they do. And that's actually really, really important, because let's say, for example, that you're competing for a sale with somebody else who has a different viewpoint of what, strategically, what you're supposed to be doing in that situation, right, and so you have sort of competing strategies and you're executing strategy and they're executing strategy. Be If you don't completely understand strategy, be number one, you can't sort of understand what about that might be good that you should incorporate into your own life. But also, even if you decide that it's not a particularly good strategy, really deeply understanding what they're doing and why they're doing it is going to help you to more effectively execute against it. Yeah, actually beat it better, because you're actually taking the time to actually understand it, not in a straw man way,...

...in a really dismissive way of that's just dumb, but to actually really take a deep dive on understanding why that person is doing what they're doing, and that's actually incredibly helpful to you. I completely agree. The last final thing I will say, because I just listen to this podcast and it was a different podcast about a different cognitive psychologist actually who was talking about because I just think it's fascinating that I agree with you that you sort of you want to get a tribe of people and sometimes your spouse is that person that helps you pursue and seek objective reality. Then that runs counter to some of the work that other cognitive psychologists have done, which says that the objective reality that humans perceive is it's like a user interface. It's a construct for the world, but the world doesn't actually exist the way that we perceive it. It's a construct that's been developed that's evolutionary successful. So I don't know. Anyway. So actually I completely agree, and I think that that's the point, is that if our goal is to because what we're doing in terms of our brains, as we're always modeling the world, we have a model of the world. We're not seeing the world as it is objectively true. We're interfacing with the world through this model that we have of it, to our experiences of it. So actually fully recognizing that is what causes you to go and find a really good decision pod and say we want to try to come up with the most accurate model of the objective truth. We want to try to get to what's objectively true as much as possible instead of just going around and believing that the things that are going on are in our brain are right. Yeah, so that requires an open mindedness, because what that means is that if you just go around and you think, well, what I'm seeing is the objective truth, then when someone sees the world differently than you, you just automatically think they're wrong because you don't recognize that there's an interface between you and the world. Right, so you just think, well, I just see the world for as it is and if you don't see it the same way I do, then I don't know what's wrong with you. You're blind or dumb or something. But once you recognize exactly what you said, which is that we're just operating off of a model of the world, then what you recognize is that well, other people have different models and my model is not a hundred percent correct. I'm just trying to get it as close as possible. So I should always be trying to calibrate this and I should always be listening to other people who view the world differently than I do, with an open mind, because maybe they have something more right than I do. Yes, I agree with you. The only time I probably disagree is if you've done all of the work and all of the thinking and you have what you believe to be a you know, you've done as much work as you can, you have a model that you feel is accurate. There are still people that disagree. You've considered their opinion and they're still wrong. I'm obviously speaking of politics, but you know, at that certain point, you know like there's I can't be both sides on everything. You know the world, the Skuy is blue. I have to believe that. Yes, so you know what. So I think that that goes to strong convictions, Lucy Lee held. So when we're making our decisions, we want to make them in a strong fashion, saying I've done the work, this is the best information I have right now, and so therefore I can decide off of this, but always sort of be open minded to listening to the other side. Now here's where I don't listen to the other side when I don't think that the other side is engaging in a genuine way, and in politics I think that that happens a lot, right. So I think that people are just engaging not for the purposes of seeking truth, but for the purposes of causing controversy or being provocative or you know, whatever it might be, and then then I'm not going to really consider what you're saying very much because I don't think that you're engaging in an authentic way. Right. So, as long as you authentically believe the thing that you're believing and you authentically think that this is true and that this is what's best for the world, I will always engage with you. And even if I've come to the conclusion that I'm really sure that I'm a hundred percent right, I still think that I'm supposed to really try to understand your side because, first of all, it helps me have compassion for you and not think that you're a bad person just because you believe something differently than me, which I think is actually really important. The other thing is that it really helps me to sort of decide in relation to you right, like if we have competing values, I need to think about that. If I want to execute on the things that I believe are right, I need to really understand genuinely why you disagree with me so and then again. And the other thing is that sometimes I might change my mind a little bit through that re engagement. So I'm always sort of trying to do that. That being said, I don't waffle when I go vote right because at the moment that I vote, I'm voting based on the opinions, in the beliefs that I have right now and who I think is going to execute on those opinions and beliefs the best. And I do that with absolute conviction. And then after that I'll engage with someone who voted for somebody else to try to understand why. It makes a lot of sense. Annie. So the book is called thinking in bets. What is the rest of the title? I don't have any murder decisions. When you don't, when you don't know all the fact, that's exactly so. Every decision, every do you have a preferred as there like a bookseller where you get a higher commission or something like that, or just happends on you know, I don't know, like just go go to your local bookstore at, you know, Amazon. I'm...

...a big supporter of Indie book so I love it when people support in the independent bookseller. So, you know, if you can do it that way, that would be amazing. There's actually a cycled in new book. The best way to engage with me to a social media is on twitter at Annie Duke. I'm actually incredibly active. I create all my own content for that. And then another way to engage with me is I have a newsletter. If you go to my web site, Annie Dukecom, you'll see our chives of the newsletter, and the newsletter is actually basically just executing on what we talked about today. So taking events, current events that are either, you know, things in science or, for example, politics or technology or whatever it might be, in really applying this kind of framework to thinking about what's going on in the world. Like I say, you can read before you buy, before you subscribe, but it's about two to three thousand words a week that you'll get for me, really kind of practical application of this kind of thinking. The other thing is that there's a contact forum on there and if you write me, I will respond. I don't think that I haven't responded to anybody, but I mean I try. I try to respond to a hundred percent of the people who write me because I love hearing from people who read my newsletter, who who've read the book, because that's the best way for me to learn and to understand, you know, what people are thinking. And I've also gotten some really cool pointers to other really cool sources of information from people who are readers of mine or different just different thoughts that they've had, and so that's a really fun thing. That's fantastic. So listen. Thank you so much for participating and for joining us today. The book one last time is called thinking and bets, making smarter decisions when you don't have all the facts and he thanks for joining us. Thank you so much for having me. Take care. He folks, this is Sam's corner. Was Really honored and excited to have any duke on the show. Any, as we know, has one multiple world series of poker bracelets use on the celebrity apprentis. But most importantly, she's a trained cognitive psychologist and studied at the University of Pennsylvania so incredibly insightful, and her new book is called thinking in Bets and the whole point of it is that we have to understand that decisions are bets on the future. I think that's a really interesting way of thinking about it. And Life is probabilistic and when you're resulting, which is the phrase that I think they use in poker, which is focusing inordinately on outcomes, you're not getting the whole picture. When it comes to sales as specifically, comes down to forecasting accuracy as an important determinant of the efficacy and success of our work process and of our sales process, and Annie mentions that under forecasting and over forecasting are both equally bad. So we need to diagnose and dissect the process. Even if, you know, we say the deal is going to come in for a hundred grand and it comes in for two hundred grand, yeah, that's a great upside surprise, but we have to understand why we were inaccurate and we also have to not in ordinarily focus on the outcome, because the outcome is a result of a series of probabilities. We have to sort of understand statistics and focus on the most probabilistic outcome, understanding that probability means that sometimes it's not going to go our way. I think it's a really helpful framework and it helps keep us and keep our perspective in line as we go through the world of sales. I also think she talks about assembling a tribe of people that are going to help you pursue objective reality, which is a really a way of saying people that are going to challenge you on your assumptions, but in the spirit of pursuing truth, not in the spirit of, you know, sort of personal denigration, and for a lot of us, like me, my wife does that. But you know, get in a company or a culture where honest, direct feedback, not about you as an individual but about the process, helps you challenge your assumptions, change your mind and update your perspective. So that has been Sam's corner. It is perhaps a more cerebral episode, but one that I really enjoyed and I really think the book is Great. Now to check out the show notes, see upcoming guests and play more episodes from our incredible lineup of sales leaders, visit sales hackercom and head to the PODCAST TAB. You'll find us anywhere podcasts are found. If you enjoyed the episode, please share with your peers on Linkedin, twitter or elsewhere. Always unclear what that word in the copy means. I guess facebook. Maybe write a handwritten note and send it in a letter. If you want to get in touch with me, find me on twitter, say m F Jacobs, or on Linkedin at Linkedincomm F Jacobs. Once again, big shout out to our sponsors. Are Call, your advanced call center software, complete business phone and contact center, one hundred percent natively integrated into any CRM and outreach a customer engagement platform that it helps effectively and efficiently engage prospects to drive more pipeline and close more deals. Now I will see you next time.

In-Stream Audio Search

NEW

Search across all episodes within this podcast

Episodes (386)