Aug 24, 2021
Imperial March Major Key Version
Ted Lasso Dart Scene(FOUL LANGUAGE)
Episode Timeline:
Episode Transcript:
Rob Collie (00:00:00):
Hello friends. There was an old Monty Python movie titled And Now
for Something Completely Different. I think that's an excellent
tagline for this show, and today's guest, Adam Harstad. Nominally,
Adam, is a fantasy football writer and analyst, but I think you'll
see that that title or really any succinct title really struggles
to describe and contain the human being that is Adam. A better term
for him might be something like curious adventurer, because that
really seems to be the way he approaches everything.
Rob Collie (00:00:34):
We went in a lot of weird and nerdy directions, as you might
expect. Yes, of course, we talked about football. So if you're not
into football, well, you can tune those parts out. But even there,
when we're talking about football, we're really talking about the
fundamentals of how to think and specifically how to think in the
face of uncertainty. We also talked about the differences between
sports analytics and business analytics, which of course are
significant. And we revisited an old favorite topic that of Nate
Silver versus elections.
Rob Collie (00:01:04):
I learned a lot of new vocabulary along the way, and I really must
say that I felt pushed in this conversation in a good way. He was
pressing us to be better, unintentionally, politely, but that's
what happens when you have really intelligent, vibrant, capable
people in your conversation. We all should be seeking out the Adams
in our life because they pull us forward on that journey of
improvement. And he's an entertaining fellow too, so let's get into
it.
Announcer (00:01:35):
Ladies and gentlemen, may I have your attention please.
Announcer (00:01:39):
This is the Raw Data By P3 Adaptive podcast with your host, Rob
Collie and your cohost Thomas Larock. Find out what the experts at
P3 Adaptive can do for your business. Just go to p3adaptive.com,
Raw Data By P3 Adaptive is data with the human element.
Rob Collie (00:02:03):
Welcome to the show. Adam Harstad, how are you, sir?
Adam Harstad (00:02:08):
I am super happy to be here today.
Rob Collie (00:02:10):
Well, I appreciate that. You're a really good sport. Talk about
just blindly wandering into something like a volunteer. No one has
had as little context for coming on this show, as we've given you,
we've really given you nothing and you're still just all smiley,
ready to go.
Adam Harstad (00:02:27):
Yeah. We got a family motto for our little kids, we wanted to
create a sense of identity. This is who we are as a family and our
family motto is Harstads try new things, so I'm pretty much up for
whatever.
Rob Collie (00:02:39):
Damn, that's how this came about is I saw a tweet from you a while
back that said, "Hey, if you're looking for a podcast guest..." You
had this long list of things that you were down for. And I'm like,
"I'm going to take him up on it. Let's see if he actually means
it." And you totally did.
Adam Harstad (00:02:54):
Absolutely. Yeah.
Rob Collie (00:02:55):
I know you better than you know me because I've been following on
Twitter and actually elsewhere for a very, very long time, not
super closely because I'm not really that fanatical about fantasy
football, at least compared to what I used to be, but you're a data
guy. There's no two ways about it. And also, bring that humanity,
there's so much creativity and that right brain stuff going on at
the same time. And that's, to me, where those two worlds meet is
just golden. I love that. Let me explain to the audience how I know
of you.
Rob Collie (00:03:28):
And at some point it's only fair, I suppose we turn the tables and
Tom and I explain to you who we are. There's random internet
stalkers as far as you're concerned. Right? I first discovered you,
I believe on the old Footballguys forums early 2000s and you were,
I believe SSOG.
Adam Harstad (00:03:36):
Yeah.
Rob Collie (00:03:36):
So I was right.
Adam Harstad (00:03:46):
That was back when I was young and still somewhat of a tool, which
thankfully I've outgrown.
Tom Larock (00:03:51):
What's SSOG?
Adam Harstad (00:03:53):
Yeah. Speaking of growing out of being a tool, I got on the
internet when I was 16 years old and I needed an email address and
Hotmail was the thing at the time. So I signed up for a Hotmail
account and being 16 and extremely full of myself. My first email
address was somesortofgenius@hotmail.com because that was the joke.
I'd say something and people would say, "What do you think you are,
some sort of genius?" Then I could respond back that, "Yes, I do
think I am some sort of genius."
Rob Collie (00:04:20):
I do. Exactly.
Adam Harstad (00:04:22):
That lasted for a year or two and then I realized, this is a little
bit much, even for me, I'm all about owning who I am, but this is a
little bit much even for me. Then I got into the University of
Florida, so I changed it to some Gator, which was relatively
inoffensive. And then just over time, it had gone through
multiple-
Rob Collie (00:04:41):
Relatively.
Adam Harstad (00:04:42):
Relatively, absolutely. Depending on where you're from, depending
on who you're asking. Over time, it went through multiple
iterations where the commonality was just the initials and so, it
became a running joke. People ask what SSOG stands for, and at one
point it stood for stuff and then after a while it just stood for
whatever you wanted it to stand for.
Rob Collie (00:05:00):
Now you're just SSOG.
Adam Harstad (00:05:02):
It's a testament to the power of path dependency. And I think, if
you ask other questions about other elements online profile, it's
all the same thing. It's a reminder to myself that where we are is
largely a function of where we started and the path we took to get
here.
Tom Larock (00:05:16):
That's deep.
Rob Collie (00:05:17):
Yeah. It's a concise summary in a way of like everyone we've had on
this show, we've yet to have a guest on this show whose career was
a cold shot. We're still looking for that person who set out to be
something in high school that they knew what they wanted to do with
their lives and they went out and they did it and they love it and
they're successful at it. We're still looking for that guest.
Adam Harstad (00:05:38):
You should meet my wife. That's her in a nutshell, which is-
Rob Collie (00:05:41):
Really?
Adam Harstad (00:05:41):
It's always amazing to me. I can't fathom that sense of who you are
at that young of an age. I still have very little sense of who I
am. I'm still working on it, but she's always known.
Rob Collie (00:05:52):
Wow. That's got to be really interesting, as a close up
example.
Adam Harstad (00:05:58):
Yeah.
Rob Collie (00:05:59):
My initial reaction is, wow, that would be really invalidating for
me, but then I recovered. I'd be okay. I'd be okay. I'd be okay
with my zigzag path.
Adam Harstad (00:06:06):
I've managed to survive for 20 years now with it.
Rob Collie (00:06:10):
Footballguys forums were one of the handful of places on the
internet back at that time, where if you were really crazy obsessed
about fantasy football, which I was at the time, you could get
exposure to some really smart people in the community and their
opinions. Not everything posted on those forums was intelligent or
worth following, but it was amazing how, after a while, you, as a
silent lurker on those boards, I almost never said anything.
Rob Collie (00:06:36):
There's a handful of personalities like these seven or eight
avatars personas that you start to associate with subconsciously
like, these are competent people. SSOG absolutely was one of those.
And so now all these years later, the mystery of SSOG, the origins
of the acronym unraveled. This is the first time I'd heard that,
but you also had the crying elephant is that Babar?
Adam Harstad (00:07:00):
I have no idea what it is, to be honest. Speaking of path
dependency and everything about my online personas attribute to
that path dependency. I was on a forum and I needed an avatar and
Google image search was the brand new thing. And I'm like, "I'm
going to play around with Google image search to see what it does."
I went onto Google image search, and I typed in SSOG and I looked
at the page of the results. But unbeknownst to me, I had
accidentally... My left hand was shifted one key to the left.
Adam Harstad (00:07:29):
So I actually typed in AAOF and the crying elephant was one of the
first results and I'm like, "This is really cool." And I love that
story. That it's serendipity, that just my hand was shifted a
centimeter to the left. And now I've attached it to everything to
create this consistent online brand because this is back before
people were using real names and having a persistent online
persona. But I really like the avatar. I think it's a little
cheeky. It's a little bit fun, a little whimsical. And it, I think
is mysterious. People are always wondering who is the elephant? Why
is he crying?
Adam Harstad (00:08:04):
But I also love it because it's a complete accident, and that is a
good reminder to me sometimes when I need one.
Rob Collie (00:08:11):
You can definitely go check this out right now on Twitter, Adam
Harstad. Is that right?
Adam Harstad (00:08:16):
Yeah.
Rob Collie (00:08:17):
You're not @SSOG, but you kept the SSOG icon, the original elephant
icon, the AAOF icon as the case to be.
Adam Harstad (00:08:26):
Again, path dependency. I tried to register SSOG on Twitter first,
there is an account on Twitter registered @SSOG that has one tweet.
It says, "In a jerking mood." And that's the entire SSOG
account.
Rob Collie (00:08:41):
Are you sure that wasn't you when you were-
Adam Harstad (00:08:44):
I’m positive.
Rob Collie (00:08:45):
...half-awake one night?
Tom Larock (00:08:47):
[inaudible 00:08:47].
Adam Harstad (00:08:48):
That's why I write under my own name as opposed to a pseudonym. At
some point I probably made a transition.
Tom Larock (00:08:54):
That account is suspended.
Adam Harstad (00:08:56):
I think so. I think so. Just because it's registered in 2009,
because I was on relatively early, not super-duper early, but I've
been on for over 10 years now and that one beat me on and tweeted,
"In a jerking mood." And then just never logged in again.
Rob Collie (00:09:10):
But that describes Twitter so well, don't you think?
Adam Harstad (00:09:14):
Absolutely. Absolutely. And they did the alternating caps and no
caps. I love it. And that's why I write under my real name.
Rob Collie (00:09:20):
By keeping the elephant, I was able to connect you with who you had
been on the forums. Are you still on the Footballguys forums at
all?
Adam Harstad (00:09:28):
Not so much anymore. People think there's this grand conspiracy
that once you start writing for Footballguys, that the bosses don't
want you on the forums as much, which couldn't be further from the
truth. It's more just, people have kids, people grow older, people
have less disposable time.
Rob Collie (00:09:41):
Let's just go back for a moment. Florida Gators, I was raised with
the Gators as a religion. I betrayed everybody and went to
Vanderbilt. I was the first person in my family to break ranks, but
then you ended up a Denver specialist, right?
Adam Harstad (00:09:54):
Yeah. I grew up in Colorado. Till I was 15, I lived in Colorado and
then we moved out to Florida. I wasn't a diehard Gator or anything.
It's just, I lived in Florida. It was in state tuition. It was the
best public school in the state, so that's where I wound up.
Rob Collie (00:10:09):
Also, coincidentally, because I was raised so aggressively on the
Gators. At an age when you're just defenseless, you're just going
to take an imprint and it's going to stick. My favorite NFL team
for a very long time was the Broncos because of the same color
scheme.
Adam Harstad (00:10:24):
Orange and blue.
Rob Collie (00:10:25):
That's right. I suffered through all of those terrible blowout
losses. I used Apple II Print Shop to make a banner for the Broncos
and hung it on the front of my house in middle school just to have
them get destroyed.
Adam Harstad (00:10:41):
At least they were making it though.
Rob Collie (00:10:42):
That's right.
Adam Harstad (00:10:43):
It's hard to complain too much as a Bronco fan. There was a three
decade span where we had more Super Bowl appearances than losing
seasons.
Rob Collie (00:10:50):
Yeah. When you think about being a fan for a sports team, it's a
losing proposition, the probability. There's only one team that's
going to end on a high note, that's it. There's one team. Take the
number of teams in the league, divide one by that that's your
percentage chance of ending the season happy. I just think the
expected value of being a fan of a particular team is actually
pretty low. Even the good years where you win the championship, I
just don't know that that really makes up for the down.
Rob Collie (00:11:19):
I've become more of a sports fan for that reason. A little bit more
mercenary. Here we go.
Tom Larock (00:11:25):
This isn't true. That's really extreme. First of all, you could say
that about any sport, but what you really mean to say is only one
team and so seasoned on the wind, but there's no way you're going
to tell me that a Cleveland Browns fan didn't end last season happy
to beat Pittsburgh in the playoff game after how many shitty
seasons. They ended happy. They didn't win it all, but they're
certainly happy.
Rob Collie (00:11:50):
Think about though, how the ground needed to be prepped for them to
reach that level of happiness about mediocrity.
Tom Larock (00:11:58):
That's fair.
Rob Collie (00:11:58):
They had to have decades of misery. I just-
Tom Larock (00:12:01):
You’re talking to a Patriots fan; I know decades of misery.
Rob Collie (00:12:05):
I know. I just don't think that being a sports fan for a particular
team was a buy and hold strategy. You inherently need to time that
market if you want to win.
Tom Larock (00:12:13):
Buy and hold is interesting because as a child, my friends and I,
we would pick one of the worst teams to start rooting for because
we knew in 20 years, they'd be good and then we could say, we were
a fan way back. I'll go get my Vikings jersey right now just to
show you.
Rob Collie (00:12:32):
Any year now. There's this, and it's a well-known human tendency
that we judge experiences predominantly by the very, very end.
They've done studies where they'll play people, this beautiful
symphony and then right at the very end, they end on a sour note
and people will say it ruined the entire symphony. There's 30
minutes of music and it ends on one note. And I just think that's
the wrong way of looking at it.
Rob Collie (00:13:00):
You had 29 minutes and 50 seconds of just the sublime transcended
experience and 10 seconds of something bad and you end your season
on a loss, but the season, isn't just the last game of the season.
It's all the anticipation leading up. It's everything you do during
the course of the year. A lot of troubles in life are just a matter
of perspective and a matter of attitude.
Tom Larock (00:13:20):
I want to hear one of these now. Is it just a sad trombone at the
end?
Rob Collie (00:13:26):
[inaudible 00:13:26].
Tom Larock (00:13:36):
Because I'd just be sitting there laughing. This is the greatest
thing ever.
Rob Collie (00:13:40):
The meta of it would immediately just like crash over me. Like, are
you kidding, you designed an experiment like this?
Adam Harstad (00:13:48):
I think it doesn't resolve, you have the progression and it's
supposed to resolve on a certain note and they're probably a half
step flat. And I don't know if you know that much about music, but
if something doesn't resolve, it creates this very real physical
tension because you're anticipating it and you're anticipating and
you're waiting for it. And it's really a cruel thing to do to
someone, but at the end of the day, it doesn't change the first 29
minutes of the symphony.
Tom Larock (00:14:12):
Tomorrow night I'm going to Tanglewood to see John Williams in the
movie night. Now he's 89. I'm pretty sure he'll still be alive for
tomorrow, so that's good. But now I'm just thinking he's going to
play the Star Wars theme and at the end of it, it's just a sad
trombone. I'm just going to be sitting there laughing at the end of
all these songs. People are going to be like, "What is wrong with
that person?"
Rob Collie (00:14:36):
All right. I don't know anything about music theory. I freaking
love music. I'm always, always, always deep, deep into music and
stuff that I appreciate is usually like on the more complex end and
I don't know anything. Let's talk about this for a moment. I heard
the Imperial March one time in the major key instead of the minor
key. The minor key Imperial March is very foreboding. The major key
version sounds like this upbeat, let's go have fun. It's like a
clown parade. Explain it to me like I'm five minor key versus major
key.
Rob Collie (00:15:13):
Have we come to the point where I'm going to understand this in my
life?
Adam Harstad (00:15:16):
The simple version is minor key sounds sad, major key sounds happy,
but that's too simplistic. There are happy songs written in minor
keys. There are sad songs written in major keys. It's been a lot of
years since I have last taken any music theory courses and that's
unfortunately some of the knowledge I've lost during my life.
Rob Collie (00:15:32):
Well, that's okay. Path dependency again. Right?
Adam Harstad (00:15:35):
Right.
Rob Collie (00:15:35):
I've also heard though that it's cultural, certain cultures will
actually find the major key to be the sad one and vice versa.
Adam Harstad (00:15:43):
Yeah. Most of our experience of music is just based on historical
context and our exposure to music. What sounds normal. A lot of our
enjoyment of music is actually anticipation. There's pattern
recognition and there are certain patterns that are common through
music and the brain really loves anticipating those patterns and
then when the patterns resolve in the way that we expect them to,
the brain rewards us with a jolt of pleasure like, "I knew that was
going to happen. That was very enjoyable and pleasant."
Adam Harstad (00:16:10):
Sometimes if the patterns resolve in ways, we were not expecting
brain results us in a jolt of pleasure because this was fun and new
and exciting. I'm eager to see where it goes next. And so, our
experience in music is heavily shaped by the patterns and our
experience to music prior to that. We can listen to a Western pop
song and we're very familiar with the genre, we're familiar with
the tropes. We're familiar with, we're going to go first course
first. Maybe sometime we'll have a coda.
Adam Harstad (00:16:40):
There's the common building blocks of music and artists will change
the order of them, they'll change the way they're put together.
They'll change the makeup of those blocks, but they're building
with the same blocks, whereas we can listen to Eastern music and it
just sounds very foreign to us because we don't have that language.
We don't have that fluency in it yet.
Tom Larock (00:16:58):
It's such an interesting topic, almost like this anticipation. I
totally see that.
Rob Collie (00:17:02):
And there are some quirky misdirections in music that you wouldn't
necessarily expect, but that are incredibly fun when they spring
them on you. It's not just personality that I've been drawn to that
it kept me reading your stuff over the years. And in fact that
wasn't at the beginning at all right. It was the quality of the
insight and most of it is data-driven. Most of it is analytical.
It's not just about the evidence that you collect. It's also about
the way that you process it that I have found valuable over the
years in trying to defeat my friends and colleagues in a silly
game.
Rob Collie (00:17:35):
How did you discover your passion for data?
Adam Harstad (00:17:38):
It's funny. I came at it obliquely. I've got a very strong left
brain, classically left brain skills, math and seeing patterns and
interpolation, things like that. But I think my love of football
and my love of fantasy football has actually always been more
narratively driven. And I like to say first and foremost, I'm a
storyteller. And it's funny you say I'm a data guy. I know data
guys who think I'm one of those narrative guys, narrative guys
think I'm a data guy, I'm in that liminal space between.
Adam Harstad (00:18:08):
For me, it's all about, I have these pressing questions and I think
the questions are the interesting part. And I'm looking for any
insight, knowledge, any edge I can get to answer those questions.
And getting back to path dependency, when I was getting into
fantasy football in early, early 2000s, that was really the
unexplored space. There's a website called Football Outsiders that
was just launching and they were actually looking at data and
they're saying, let's compare every play in this situation to every
other play in the situation.
Adam Harstad (00:18:40):
If you ran for four yards in first and 10, was that a good play?
Was that a bad play? Nobody had ever really dug into it. And so,
that was the edge. That was where the edge is. And over time as
analytics has gotten to be a bigger and bigger part of the sport of
football and the hobby of fantasy football, I find often the edge
is in this, not really contrarian to analytics, because I think
analytics is good and useful and right. Analytics is a way of
approaching problems by looking at the data and that's a very, very
useful tool.
Adam Harstad (00:19:13):
But I think that there's some conventional wisdom that should apply
to analytics that has gotten lost along the way in the rush to mine
more data. So for me, it's never been specifically about the data.
It's been about the questions and the answers. And for most of my
career, the data has been the best way to reach those answers, but
it's not the only way. It's more about finding the appropriate tool
for the appropriate situation.
Rob Collie (00:19:37):
I think I'm starting to understand why I felt this remote kinship
with you asymmetric, because that's how the internet works with
lurkers and publishers. Our team is growing really rapidly at our
company and we're a remote team. We're all over the country, even
though we're full-time employees, it's a good problem to have, but
it's become difficult to keep track of who everybody is. Name,
face, okay. But what about personality? It's so hard. Yesterday we
started collecting information to make flashcards for
everybody.
Rob Collie (00:20:11):
Everyone had just to pick a "superpower" or something like that,
but not something super professional, but like storyteller. I
picked storyteller for me, folksy storytelling would be one of my,
"Look out, he's going to do it." And then the football, I'm a data
professional. There's no two ways about it. I worked on data tools
at Microsoft, I worked on Excel, I worked on power BI, I'm CEO of a
data analytics, business intelligence consulting firm. Yes, data
professional. But that all started for me in the late '90s with the
same sorts of things as what you're talking about.
Rob Collie (00:20:45):
Like how do I actually approach this game, fantasy football? How do
I approach it more effectively? I stumbled upon the original
value-based drafting article years ago, which put the different
positions into perspective in terms of relative value. And I was
just like, "Ugh" It's like the holy grail. It was this magical
unlocking moment and what followed was just reams of spreadsheets
and coding and all kinds of things. It was the only times, even
though I was a software professional already, I was already working
at Microsoft.
Rob Collie (00:21:15):
I was never really that into it. It was one of the few times that I
was actually into it was when I started doing these sorts of
things. Another thing that you wouldn't have the context for is
that on this podcast over and over and over again, we talk about
how we observe even, the value is in the hybrids of different
centers of mass. In the business world of data, it's long been like
these completely separate universes where IT was doing some data
stuff and business was doing some data stuff and they almost never
collaborated.
Rob Collie (00:21:51):
And really the software was built to create this divide in the old
days, but now there's this rise of these IT/business hybrids. And
again, it comes back to all these sorts of things that you were
saying conventional wisdom is a shortcut for saying all kinds of
things, all kinds of developed expertise and wisdom that you can't
just set aside when you're using data. You need to integrate the
two. IT can come into a business situation and bring all kinds of
technical skill, but they've got a complete blank slate when it
comes to the business knowledge of what's actually going on.
Right?
Adam Harstad (00:22:28):
Right.
Rob Collie (00:22:28):
It turns out that you can't have one or the other, you have to have
both. And that just keeps coming up on this show for good reason so
I think there's a fundamental truth that we're experiencing
there.
Adam Harstad (00:22:39):
Yeah, for sure. I think where fantasy football at is gotten a
little bit too disdainful of other forms of received wisdom.
There's this idea that what the data says is true, which is usually
the case, unless you're making mistakes, analyzing the data, which
is very, very easy to do, but there's also the sense that what
isn't coming from the data is not necessarily false, but suspect.
And as an example, there was a recent debate about whether throwing
two running backs is a positive or negative for NFL offenses.
Adam Harstad (00:23:11):
And the data guys looked at it and they said, if you look at the
expected value of every play, passes to running backs are less
valuable than passes to wide receivers, which is true, I'm sure
that's exactly what the data says. They're smart guys. They know
how to analyze it. I'm sure that they're controlled for the
appropriate confounders, various other pitfalls, but at the same
time, I'm looking at it like, that may be so, but if you look at
the best offenses of the last 15 years, disproportionately, they've
thrown to their running backs a lot.
Adam Harstad (00:23:40):
You look at Sean Payton's New Orleans Saints throw to their running
backs more than any team in the of football, and it's hard to find
any offense in history that has had as much sustained success as
the 2006 to 2020 New Orleans Saints. Bill Belichick, New England
Patriots, you have James White, Shane Vereen. They historically
have had a back whose only job is catching the football. Danny
Woodhead was there, and it's hard to look at these and say,
throwing to running backs is a bad play and yet a lot of the best
offenses in history do it a lot.
Adam Harstad (00:24:13):
A lot of the worst offenses in history do it a lot too. I don't
think it's a silver bullet, but it's hard for me to reconcile the
idea that this is just outright bad with this observation that it's
so often successful. And so, to me, the received wisdom is maybe we
have reasons to be suspect of passes to the running back, but I
can't just outright declare it bad, but that's not coming from
data. That's coming from anecdote and observation, which is a
powerful tool for receiving insight that I don't think is
necessarily reckoned with enough.
Tom Larock (00:24:45):
On that, what I would say, and I got a whole lot I want to talk
about when we talk about bias and things like that, and I just
followed you on Twitter so I could retweet your thing about
selection bias. I started fantasy football in the mid '90s,
roughly. Well, I think we all did when it was really up and coming
and no offense, but I'd never heard the Footballguys until today. I
knew the Football Outsiders and I knew a handful of websites and
this was before even ESPN or CBS Sports was offering this data, so
I've been there.
Tom Larock (00:25:16):
What you're talking about just now with the throwing to the running
backs, what hit me was all the data that you're examining is
historical and you have no idea of what's going to happen next. You
can say, everything we do in fantasy football, all we're doing is
managing risk. I have a risk of scoring zero points, how do I make
sure that I'm at least scoring something and that's all we're
doing. We're just looking for a way to make sure we're producing
some value on the field, so to speak.
Tom Larock (00:25:48):
Anyway, let's look at, if you could, as an example, before the 2007
season, when the Patriots had Moss and Wes Welker, nobody had any
idea of just how many points that team was going to put on the
board that year, because historically it just didn't happen. Maybe
there was an offense from the 40s that ran something similar, but
probably didn't have the same numbers generated. But when you talk
about Belichick and a lot of coaches in the league, everything old
is new again.
Tom Larock (00:26:20):
He just says, "All right, what talent do I have? Where would they
excel?" This type of a formation. It's not really hard. They just
look and say, "It's fairly basic. You need to do these things to be
successful. Wes, go into the slot, run that way. Brady might hit
you because Moss is just going to go de..." My point is simply, you
can do all this analysis, at the end of the day you still don't
know what's going to happen if somebody gets hurt. And to me, like
I said, it's just managing the risk of scoring zero points. That's
how I look at it each week.
Tom Larock (00:26:52):
I have a risk of scoring nothing, I can actually in this league be
negative. How do I avoid that?
Adam Harstad (00:26:57):
One of my favorite articles I wrote was actually, you guys are
probably familiar with the phrase, "The map is not the
territory."?
Rob Collie (00:27:02):
No, I've never heard it.
Adam Harstad (00:27:03):
It's this idea that a map obviously is not the territory that it
represents. It's a map. If you draw a mountain on the map, it's not
like a mountain is going to magically spring from the ground and
it's a representation. And I do a lot of historical modeling and
that's a lot of my process is I will look at historically
comparable players. Ja'Marr Chase comes in, what's the history of
highly drafted rookie wide receivers? And from that, I'll create my
baseline expectations. You need your prior and you can adjust your
prior from there, but it's good to have that strong prior to
start.
Adam Harstad (00:27:38):
But the problem is, let's say that one of the historically
comparable players for Ja'Marr Chase is Larry Fitzgerald, that's
going into the mix. If Larry Fitzgerald had torn his Achilles
tendon as a rookie and never played another snap, the comparable
players for Ja'Marr Chase would be worse, but his prospects
wouldn't be any different. Realistically, it's not like Larry
Fitzgerald tearing his Achilles would make Ja'Marr Chase more
likely to tear his Achilles.
Adam Harstad (00:28:05):
Similarly, if someone like Charles Rogers who was a colossal bust
had gone on to become a Hall of Famer, the comparable players to
Ja'Marr Chase would look better, but Ja'Marr Chase's prospects
wouldn't change at all. And it's something that I think you really
need to reckon with when you're doing this kind of historical
modeling. You need to remember that the map is not the territory.
Every player in this situation has done poorly before, it doesn't
mean that this player is going to do poorly now.
Adam Harstad (00:28:28):
But the full quote is, “The map is not the territory, but it has
similar features accounting for its usefulness." If it's drawn
correctly, it will resemble the territory enough that you can use
it to navigate and it'll get you where you need to go if it's a
good map. But the famous René Magritte has the treachery of images.
It's the picture of the pipe and underneath he writes, "This is not
a pipe." And everybody gets really mad because they're like, "Of
course it's a pipe. Of course it's a pipe."
Adam Harstad (00:28:54):
And he said, "Okay, well, can you stuff it? Can you light it? Can
you smoke it? No, it's not a pipe. It's a picture. It's not a
pipe." The map is not the territory, but it shares similar features
and that accounts for its usefulness.
Rob Collie (00:29:06):
We still need maps. Just be careful how you use them.
Adam Harstad (00:29:09):
Lewis Carroll wrote once about a country that created a map with a
scale of one mile equals one mile, but then they didn't find it
that useful, so now they use the territory as its own map and it
serves almost as well.
Rob Collie (00:29:20):
Yeah, it's perfect actually.
Tom Larock (00:29:24):
That sounds more like a Steven Wright joke.
Rob Collie (00:29:26):
Yeah, it does. Sports, data, sports analytics. I think it draws a
lot of the same personalities to it that business data draws, but
the similarities between the two as domains, as professional
activities, or even as hobbies, they don't actually share that
much. There's some really, really, really important differences
between the two. One of them is that sports is adversarial. It's a
hundred percent adversarial. It's one team versus one team or one
player versus one player. And the business world isn't like that.
The business world inherently, it's competitive, but it's much more
collaborative.
Rob Collie (00:30:09):
People you're interacting with are your customers, not your direct
competitors. And so, that changes everything. There's just so many
things that happen. Like when we were talking earlier about
throwing to the running back, for instance, one hypothesis we can
form is that if you're effective at throwing to the running back,
you maintain an information advantage over your opponent. You keep
them guessing, you're unpredictable in a way that works to your
advantage. Being unpredictable doesn't work to your advantage in
business.
Rob Collie (00:30:40):
There's nothing to be gained by, "You didn't expect that." It's
just not like that.
Tom Larock (00:30:46):
You mean every time Google kills a product, that's just not helping
them?
Rob Collie (00:30:50):
No one expected Microsoft to buy ProClarity and basically put an
end to it back in the day and that was just good fun. Kept things
interesting.
Adam Harstad (00:30:58):
Kept everybody on their toes.
Rob Collie (00:30:59):
Yeah, that's right. You never know. We might buy the number one
front end for our server and basically retire it. Anyway. They
don't do that anymore. That's the old Microsoft that did that kind
of thing.
Adam Harstad (00:31:13):
Monopoly rules.
Rob Collie (00:31:14):
Yeah, that's right. And also, software engineers running the show.
Anyway, we're not going to talk about Microsoft too much today. The
other thing about sports is that almost everything that you would
do analytically with it is predictive in nature. Whereas in
business, not everything, there's still absolutely predictive stuff
that needs to be done in business and we do some of that for our
clients. But like, let's say if I was speaking to the other
audience, the sports analytics audience, who's never worked a day
of business analytics.
Rob Collie (00:31:45):
The analogy I would give you is, what if you only found out the
score of the game, that's all you learned was the output like,
"Shit, we lost again. Bad this time. 31 to 10." But you didn't get
to watch the game. You don't get any box score and we're the ones
that use all of this incredibly fine grain data that's happening in
the business to eventually explain why things are happening the way
that they are. Even just knowing where you are, knowing what's
actually happening.
Rob Collie (00:32:12):
There's a joke in our industry, business intelligence by bank
account, which is you get to the end of the year and you're like,
"Hey look, how much money is in our account. Way to go team." Or,
“Damn. Terrible year. Everybody just grit your teeth and try harder
next year." That's reality. The lights are out by default and it's
a tremendous oversimplification, but at the same time, there's a
lot of truth to it, which is, we go around turning the lights on,
even just explaining what has happened.
Rob Collie (00:32:41):
Typically, we don't want to be like on a one-year delay to
understand what's happening, we want to know early in the month,
what you're trending towards and why, so that you have an
opportunity to make changes. It's all about forward changes. So
many differences about sports analytics versus business analytics.
And yet we're all moths drawn to similar flames. I watched the
skills that you've developed over the years as I've been watching
you, Adam, are very different than the skills that I require in the
business that I'm in on a daily basis.
Rob Collie (00:33:13):
There are outlier applications for a lot of the things that you do
in business. Absolutely, but it's like the average task, the
average analytical question in sports versus business is really
completely different.
Adam Harstad (00:33:27):
Well, and I think a lot of it too, is that in sports, the whole
possibility space is so constrained. If you have an offense at the
one-yard line, there's basically a hundred possibilities for that
play. They can gain zero yards, one yard, two yards, three yards,
so on and so forth. Whereas, if you have a startup at the business
equivalent of the one-yard line, the possibility space is just so
much larger. They can succeed, they can fail, they can succeed and
fail in old ways. They can succeed and fail in new ways.
Adam Harstad (00:33:56):
Really, everything's on the table, but sports because it happens in
a clearly defined arena of competition, under rules that are laid
out to all the participants in advance and everybody has an
opportunity to study, maybe some people are more familiar with the
rules than others, but they're all constrained by the same rules.
The world works in a very predictable and ordered way in a way that
the real world outside of sports just doesn't operate.
Adam Harstad (00:34:21):
And so, yeah. It doesn't surprise me that there's not that much
carry over because it's such an arbitrary space to analyze. People
talk about sports as a metaphor for life, and maybe it works as a
metaphor, but it's not really a great analog
Rob Collie (00:34:34):
At the heart of it, let's get metaphysical nerdy.
Adam Harstad (00:34:36):
Totally.
Rob Collie (00:34:38):
In the history of warfare, which has actually a lot in common with
sports, whether we like it or not, that adversarial nature, the
only constraints are the physical world. At any point in time,
there's never been impervious armor. The armor of the day has
always been vulnerable to the weapons of the day. There's never
something that, you could just build, it was invincible. Of course,
armor of today probably could stop everything from 2000 years ago,
but at that particular point in time, and why is that? The same
materials, right?
Rob Collie (00:35:09):
Same materials, same manufacturing, same technology is available to
both weapon makers and armor makers. It's just that the weapon,
only the weapon "knows" where it's going to hit. There's an
information advantage that the weapon has. The armor has to be
prepared for everything, but the weapon gets to pick whatever spot
it happens to hit. There's an information advantage for the weapon
and like in football or really in sports, going back to the thing
we were talking about, the being unpredictable, but still effective
seems to be like a crucial element of a well-designed offense.
Rob Collie (00:35:42):
And guess what? There've been very, very, very few defenses in the
history of the NFL that are capable of shutting out their
opponents, very similar to the armor and weapon thing. Information
conveys a very, very specific advantage in adversarial situations.
Of course, information also conveys an advantage in business. I
can't put this topic down. You seem smarter than I am, so let's see
what you think of this.
Adam Harstad (00:36:05):
I seem that way, for sure. SSOG, man. Projection. It's a careful
projection of certain appearances. Yeah, I'm thinking about it. I
think a lot of it's probably an incentives issue too. I don't know
that there's the incentive to build impenetrable armor, wholly
impenetrable armor like there is to build unstoppable weapons. If
you look at history and you're mentioning that war is very zero
sum, the societies that survived and the cultures that survived are
the ones that aggressively expand it.
Adam Harstad (00:36:38):
If you had a hypothetical city state that just wanted to just sit
in their city and never expand territory, eventually they're going
to be overrun. And so it's almost like the rules of the conquest
favor the weapon making over the armor making. You need armor
that's just good enough to allow you to use the weapons that you've
created. I completely agree that information asymmetry, especially
in zero sum competitions is very important. One game theory concept
I go to a lot is called a blotto game.
Adam Harstad (00:37:11):
And a blotto game is a game between two people where you each have
a certain number of resources. Say you're a general and I'm a
general and we're contesting three battlefields. And we each have
five units that we can send to those battlefields. And whoever
sends more units to a specific battlefield is going to win that
battlefield. And whoever wins more battlefields is going to win the
war. You can allocate your units however you want.
Adam Harstad (00:37:33):
You could send all five units to one battlefield, which will
guarantee you that you conquer or at least hire that battlefield,
but then you're leaving the other two battlefields undefended and
the other general will almost certainly win the war. That's
definitely a losing strategy you could send and there's also, you
could do three, one and one, you could do two, two and one and you
can evaluate all the possibilities and some of them will be Pareto
optimal, some of them will be completely dominated by other
strategies.
Adam Harstad (00:37:58):
There's a strategy that will always lose to another strategy or at
worst tie it. And you can study these blotto games and there's
variants on them. Say one side has more resources than the other.
Say that some battlefields are worth double points. It's different
if you're just going to play at once or if you're going to iterate
it over and over again. But the best advantage you can have in a
blotto game is having more resources than the other guy. If you
have 10 units and I have two units, it's going to be trivially easy
for you to win.
Adam Harstad (00:38:26):
And then the second best advantage you can have in a blotto game is
knowledge of what the other guy is doing. Because if I know your
strategy, it becomes very easy for me to just pick the strategy
that defeats it. And in zero sum endeavors, fantasy football, among
other things. I find a lot of instances that if you look at it, it
really is just a more expensive version of the blotto game. Even
football itself, I would call something of a blotto game. An
offense has five receivers. These are its units. And it's trying to
capture territory.
Adam Harstad (00:38:58):
It's trying to get a receiver into an unoccupied space by the
defense so that it can complete a pass and progress the football,
and then the defense has its seven defenders and it's trying to
defend this territory. And really there are usually on most plays
seven guys in coverage versus five guys in pass patterns and yet
the offense wins more than the defense. And the big reason why is
because the offense knows what it's going to do, it knows where
it's going.
Adam Harstad (00:39:23):
It has that informational advantage and especially quarterbacks,
Peyton Manning famously could read the defense before the snap and
know what it was going to do. And so, it was much easier for Peyton
Manning to win that particular blotto game than an average
quarterback, because he had this fore knowledge of how the defense
was going to be deploying its troops, what battlefields it would be
contesting and which battlefields it would be leaving relatively
open.
Rob Collie (00:39:43):
Have you been exposed to the concept of reactive armor?
Adam Harstad (00:39:47):
No, I haven't.
Rob Collie (00:39:48):
To me it's genius and it's only something that the Russians would
have come up with and they had it back in the Cold War. If we look
at the information advantage that the giant metal dart fired from
NATO tanks has, it only has to worry about the place where it hits.
All of its work is concentrated on this little square centimeter or
whatever of armor that it impacts. And all that other armor is
useless. All that weight that the tank's been carrying around,
except for this one little column of metal that's in front of the
dart, all the other stuff doesn't participate in his interaction at
all.
Rob Collie (00:40:27):
And the Russians said, that's the problem, so when we get hit with
a dart, we're going to have an explosive go off within our armor
and slide the armor sideways in that section so that it's
constantly feeding fresh metal into the interaction. It's crazy.
And when the iron curtain came down, when the Berlin wall fell and
NATO took some modern Soviet tanks out and tested them to know if
the darts would go through, this armor stopped everything. And what
was the predictable NATO response given the way that we think?
Rob Collie (00:41:04):
Just make the darts longer. But we see this, the defenses that are
able to react like in football, more nimbly and quickly to what
they see in front of them are the ones that are... There's no
overpowering the offense when the offense has the information
advantage, even though you have more players.
Adam Harstad (00:41:25):
There's one aspect of the defense, the pass rush actually is the
whole situations are reversed when it comes to defending the
quarterback. It's the offense that is on defense so to speak,
that's reacting. And it's the defense that gets to choose, we're
going to pass, rush these players. They don't know who's going to
be rushing. We're going to rush from these angles, we'll be doing
stunts to try and create confusion. So, for the most part, on a
grand level, of course, it's the offense that's reacting.
Adam Harstad (00:41:48):
But it's interesting that there's also that sub competition at the
same time where the defense has the first mover advantage. And if
you look at player salaries, they largely reflect that. The players
who get paid the highest are the wide receivers who have the
informational advantage against defenses and then they're the
defensive ends who have the informational advantage against
offensive lines.
Rob Collie (00:42:11):
Interesting.
Tom Larock (00:42:14):
Every sport is action reaction.
Adam Harstad (00:42:17):
Yeah.
Tom Larock (00:42:18):
All right. Every sport is action reaction. What I find interesting
about football is, and it's probably because it's easier to see a
mistake in football, I think than in other sports. And to me,
football is always, maybe it's because I hang around Patriots more
often, and this is why I watch it's about just limiting mistakes.
You already know, Peyton Manning, he's already figured it out and
he's already got a play. He already knows what to do to get a few
yards. Your job is to make sure he only gets those few yards.
Tom Larock (00:42:51):
So you limit the mistakes that you're about to make, and you keep
the play in front of you. And I think when it comes to other
sports, I think baseball, the obvious mistakes, somebody falls
down, but there's a lot of base running mistakes. Like I didn't
take the extra base like I could have. Things like that, that you
don't really see unless you're really in tune with the game.
Basketball is the same where mistakes are made. The obvious one, I
dribbled off my leg, but the not so obvious one is you didn't set a
good enough screen.
Rob Collie (00:43:18):
Well, the consequences. The consequences are so high in football. A
mistake in football can have ungodly implications. Whereas the
worst thing that can happen in a basketball play is the other team
scores, three points, four points. They score something like 4% of
their total for the game and then the slate is clean again.
Adam Harstad (00:43:34):
Well, by that standard it would be soccer. That's the worst because
the consequences of mistakes in soccer are catastrophic.
Rob Collie (00:43:39):
Yes. I concur. For example, headbutting the opposing player in the
final minutes of the World Cup final, because he insulted your
sister or something. To me that is the most leap off the couch
moment in all my sports watching history-
Tom Larock (00:43:57):
Really?
Rob Collie (00:43:57):
...was when Zidane headbutted that guy and got ejected.
Tom Larock (00:44:00):
Not when Tyson chewed Evander's ear off?
Rob Collie (00:44:05):
No, because you know what? Two whole nations hung in the balance at
that moment. Who cares about two guys punching each other? This is
national pride.
Tom Larock (00:44:15):
He wasn’t punching him, Rob, he chewed his ear off in the rain.
Rob Collie (00:44:19):
That's fine. Zidane's headbutt still takes it for me.
Tom Larock (00:44:22):
No, that's fair. Yeah, that was a remarkable moment.
Adam Harstad (00:44:26):
I Agree. The consequences of a mistake. What's the max mistake?
Tom Larock (00:44:30):
In business it's the same thing because you want to mitigate risk.
There's a lot of parallels where you're going to do these things
and you know that say, Excel, isn't going to ship in time. What's
the risk that we're after falling behind other competing products?
We can do three of the four things in order to ship on time. The
one thing that we're not going to do is going to be the least
costly for whatever, but that's how the world works. It's not
always about being perfect, it's about making sure that your
shortcomings, your mistakes are limited.
Rob Collie (00:45:04):
Adam, you said that at the beginning that you have a saying at your
house, Harstads try new things. I don't think we've ever really
codified it, but we like to say Collies play offense. In business,
it makes sense to play some defense for sure. Defense is also like
the luxury of businesses that are already successful. On the way
up, you can't defense your way to the top. You've got to offense
your way to position of value with your customers.
Rob Collie (00:45:34):
We don't have to talk about it as an adversarial conquest thing.
You have to create something in order to stand out, you have to
create something in order to change the world. I think business is
inherently again, until you reach the top and become the monopoly
and then suddenly you're like, 'Yeah, build walls, build walls."
Except for those rare cases, it's always about create. It's always
about leaning forward and being on the creation side, the offensive
side of the game. And even as a life philosophy, members of my
family, my extended family, they play the game of life.
Rob Collie (00:46:06):
You could capture their strategy as, "Avoid disappointment. Don't
do that. Don't live like that." In football if you wanted a
quarterback that never throws any interceptions, I'm your guy. You
can hire me. I'll go out there and spike it into the ground every
single play and collect my money. You want a zero interception
quarterback.
Tom Larock (00:46:27):
I'm the guy.
Rob Collie (00:46:27):
I am the optimal quarterback for you. I'm pretty sure that even at
47 years old, I can get the ball into the ground before the defense
gets to me.
Tom Larock (00:46:35):
Just three QB sneaks.
Rob Collie (00:46:37):
No, no, no sneaks. That's dangerous. No, no. I'm not going to be...
Mm-mm (Negative). I'm going to be running away from everybody and
throwing it into the ground.
Adam Harstad (00:46:44):
It's really interesting you bring that up because there's a lot of
research now that everybody used to think of sacks as a defensive
line offensive line thing, but there's a huge body of research at
this point that sacks really are a risk tolerance thing for a
quarterback. Some quarterbacks will accept more pressure trying to
get a playoff and others are more risk adverse and they'll get the
ball away quicker. And so, the famous example, I use at this is,
there are two quarterbacks in history who have had two seasons in
the top 10 all time in soccer percentage.
Adam Harstad (00:47:13):
One of them is Dan Marino, very famously able to get the ball out
quickly and avoid sacks. And the other one is Joey Harrington, who
is very famously one of the biggest quarterback busts of all time.
Basically nobody has avoided sacks as well as Joey Harrington
avoided sacks. And that wasn't a good play.
Tom Larock (00:47:30):
Because he's not playing?
Adam Harstad (00:47:31):
Because he's not playing. He's unwilling to accept any risk at all.
As soon as the pressure got anywhere near him, the ball is in the
stance and his offense was horrible because they couldn't gain any
positive gains because he was unwilling to accept that risk. And he
wrote a really interesting piece after he retired about how, "My
career was not a failure."
Adam Harstad (00:47:50):
And he talked about, "When I was in Detroit, I was asking my head
coach for permission to take risks. And I just never felt like he
was behind me and he was willing to let me fail. And so I was
always too scared of something bad happening to really try to make
something good happen." But yeah, if you look at the quarterback
performances that correlate to quarterback quality, things like
interception percentage are basically meaningless.
Adam Harstad (00:48:11):
At the very extremes, good quarterbacks tend to throw slightly
fewer interceptions than bad quarterbacks, but it's not a very
pronounced difference. The big difference is that good quarterbacks
make a lot more positive plays. They get a lot more yards per
attempt, they throw a lot more touchdowns and sometimes they'll
have some negative plays mixed in there as the risk that they're
taking. It's that level of risk that they're willing to tolerate to
get that offense.
Rob Collie (00:48:35):
Let's go back to Joey Harrington and Detroit for a moment because
when we were earlier talking about let's model the history of
highly drafted rookie wide receivers, and I was going to make the
joke, but it's not just a joke, there was some truth to it. Well,
part of the decision tree needs to be well, are they being drafted
in Detroit? And they did. They burned a lot of draft capital on
highly touted players. And only one of them, as far as I know,
really even came close. Megatron. He was the real deal.
Rob Collie (00:49:05):
But all those other people, how many people did they draft in the
first? It was at least two or three, right? And none of them went
anywhere.
Adam Harstad (00:49:11):
I think they took four receivers in the top 10 in five years. I
think that Charles Rogers, who was one of the biggest busts of all
time. Roy Williams, the wide receiver, who was okay and they traded
him to Dallas and they recouped a lot of his costs. Mike Williams,
who was another big bust and Calvin Johnson. Four in five years,
and it became a running joke. But the thing is the fourth one was
Calvin Johnson and he was probably the best pick of that era.
Adam Harstad (00:49:38):
There's a lot of sunk cost fallacy that we've tried this three
times, we've tried drafting wide receivers and it hasn't worked
out. I think a lot of people would have cut their losses and said,
"I won't deal with that ridicule of picking another one." But to
their credit, they stuck to their board and they said, "This guy is
the best prospect on the board." And a lot of drafts success and
failure, I think far more than people would believe is just
randomness.
Adam Harstad (00:50:02):
Any team that had been on the clock at the number two pick when
Charles Rogers was there would have taken Charles Rogers because he
was the best wide receiver prospect, maybe one of the best ones of
all time. Possibly the best-
Rob Collie (00:50:13):
Maybe.
Adam Harstad (00:50:13):
...wide receiver prospect of all time. And everybody thought that.
It wasn't a contrarian pick. That was the chalk pick and it's just
Detroit happened to be the team that had that number two pick.
Rob Collie (00:50:23):
Yeah, but I wanted to explore the possibility that he might've
succeeded elsewhere. We were just talking about Joey Harrington not
having the support of his coaches. There's clearly been something
culturally rotten with the Detroit franchise for a while, or at
least that's an easy narrative to paint and we don't actually know,
we don't get to run the experiment. The Charles Rogers experiment.
We only got to run it once with one set of variables and that's in
Detroit in that era. I agree with you though, on the sunk cost
thing.
Rob Collie (00:50:53):
Being able to distinguish the difference between a sunk cost
situation and a pot committed situation is one of the hardest and
most valuable things to do in life. Sometimes you can cut your
losses and that is the right move. It's silly to think of it that
way when you're making picks, it would be really silly to think
that wide receivers are terrible, we should never do that again.
That would be a bad strategy. Let's drill in for a moment.
Rob Collie (00:51:18):
In poker, there are decision points in a game where you only have a
20% chance of winning the pot, but it still makes sense for you to
continue and invest more money because the value of what's in the
pot is so great, the expected value is positive. The one time you
win out of five or whatever, will pay off so heavily that it
doesn't matter. It pays for all the losses. Sometimes you have sunk
costs.
Rob Collie (00:51:42):
Sometimes it's like, no, no, just because I threw $80 in there
doesn't mean I should throw another 20 and distinguishing between
those two circumstances, I think there's a lot of that in business,
a lot of that in life is like, which situation am I in? Am I on the
road to potential success and I need to put that next piece of
incremental investment or effort in? Versus, no, let's not keep
chasing a bad bet. There's no formula for this. There's no formula
in life for being able to distinguish between the two, but the
better you get at it, the more successful you're going to be.
Adam Harstad (00:52:15):
And that's another situation where poker, it's much easier because
the possibility space is so constrained. In poker, you can just do
the math and you can say, I'm pot committed or that's all sunk
costs. There's a right answer in poker. And that right is knowable.
The range of possible outcomes in investing a stock is it's worth
nothing or it's the next Facebook. The range is so huge. There's no
way to know in advance. That's another one where I like these
contrived arbitrary games with their clearly defined rules, because
I think they set themselves up to analysis so much more nicely.
Rob Collie (00:52:49):
Well, and they also set themselves up to be great metaphors for the
more complicated playground of real life. I think they very purely,
sometimes anyway, very purely participate in the forms that we see
with infinite noise in the real world. It helps me as a learning
tool, but yeah, in order for it to even exist, you have to
constrain it like you do in sport. We can't let this go without
mentioning the haiku. My personal favorite interaction with Adam on
Twitter was when we were, I think you started it, Adam. It wasn't
my idea. I participated.
Rob Collie (00:53:21):
I made my one contribution, but there was a whole thread years ago
of football haiku, which is exactly what you expect. An
analytically minded football Twitter account, we're going to start
writing haikus. That's exactly what everybody would expect.
Adam Harstad (00:53:38):
That's why I say, the narrative guys think I'm a data guy and the
data guys think I'm a narrative guy. And I always say, "I'm just a
storyteller."
Rob Collie (00:53:45):
You're doing it right. If either extreme is painting you into the
other corner, I think that's a sign that you're over the target.
You're taking flack from both sides. Well, that's where you need to
be.
Adam Harstad (00:53:56):
In general, I tend to think the things I believe are correct,
because if I didn't think that I wouldn't believe them.
Rob Collie (00:53:59):
That's really funny. Isn't it? And that's strange.
Adam Harstad (00:54:03):
Yeah. It's weird how that works out.
Rob Collie (00:54:05):
You could almost extrapolate that to everyone. Everyone works that
way-
Adam Harstad (00:54:09):
It's true.
Rob Collie (00:54:10):
...which is really frightening, and validating at the same time, I
suppose. It's all about uncertainty, navigating uncertainty. I
think that's the thing that we were talking about with the NFL
draft and things like that. No one really knows anything. No one
knows what's about to happen. I love that at P3, now we're up to a
14-person fantasy football league and advertised it to the team as
Basket of Virtual Volatile Assets, BVBA, that's what we're really
calling it. That's the game we're playing. it happens to use
football, but it's not really fantasy football. It's BVBA.
Rob Collie (00:54:47):
And most people who are playing with us have played before, but
there's a few people who haven't and so, I'm putting together a
primer for them. My PowerPoint deck is in progress with the
animations and everything. And the very first slide in this is
going to be a collection of predictions for the year, rankings,
projected rankings of players and which teams are going to be good
and all that. And then there's the giant word, ha-ha, over the top.
It's all wrong. It's not even going to survive a week. And every
year crazy things happen.
Rob Collie (00:55:20):
A player that we're not even talking about right now is going to be
crucial to fantasy football success this year. A team, it isn't
even a trendy team to be good out of the 32 teams. No one's picking
to certainly improve is going to improve massively. It is such
uncertainty and when I finally arrived at, in terms of my
relatively successful, I did not want to play fantasy football in
the league with Adam.
Rob Collie (00:55:46):
I'm not going to be that guy, and home leagues level of success,
home and work league type of success is that it's just going in and
just saying, "Look, my entire strategy is going to be based around
the idea, the admission that none of us know anything." And you
actually behave differently when you accept that. You make
different decisions. It's not just about expectations.
Rob Collie (00:56:12):
You decide differently, and one of the things that I've seen that
I've been doing for a long time and it's been good to me when I
start listening now, like this year, I've really been listening to
a lot of podcasts about fantasy football, it's the first year I've
ever done this. And it's so amazing to see how widespread this
notion is, that we really primarily only care about the best
possible outcomes. When you're playing a zero sum game where you're
one out of 12 or one out of 14, in order to win, you need
exceptional things to happen in your favor.
Rob Collie (00:56:42):
And you need to maximize your chances of getting those exceptional
things. To use a baseball metaphor, singles and doubles aren't
going to win you the title. You need to be hitting triples and home
runs and the nature of football, plus, I find this emotionally
satisfying, it's like prospecting for gold. I'm not going to be
drafting players or picking players who I know to be predictably
mediocre. I'm going to be drafting and selecting for best possible
range of outcomes and if they turn out to not be exceptional then
I'm going to get rid of them.
Rob Collie (00:57:12):
This isn't genius, this isn't some new strategy. I hope that people
at our company who are going to be competing in fantasy football
against me aren't listening. but some of them probably are. I used
to think that I might actually be able to pick players. You can't
predict what's going to happen in an NFL season. It's crazy. And I
love that.
Adam Harstad (00:57:29):
Real quick, I want to push back. You were saying you put up the
list of rankings and then you were saying, "Ha-ha, they won't even
last a week." Actually, empirically I found pretty consistently
looking at it over the last 10 years that preseason ADP average
draft position, the order that players were drafted in before the
season better predicts rest of year performance than current year
to date performance does until about week four. And that's when
they retract parody.
Adam Harstad (00:57:56):
They will last a week. They'll last approximately four weeks, is
one way to look at it. Or I guess another way to look at it is if
you just drafted off of last year's order of finish minus guys who
retired or who are clearly injured, things like that, that will be
more predictive of year to date production up until about week
three. My stance is everything we do in the off season, all of the
analysis, all of the... Everything buys us one more week before
we're wrong. If we did none of that, we could make it till week
three before your due date was more powerful.
Adam Harstad (00:58:28):
But by doing all of that, it buys us one extra week before we're
wrong.
Rob Collie (00:58:33):
That's amazing.
Adam Harstad (00:58:34):
In that sense, it does get you one extra week.
Rob Collie (00:58:36):
I just got snared on my own flare for hyperbole.
Adam Harstad (00:58:42):
I just figured you would like to put some numbers to it.
Rob Collie (00:58:43):
And I wouldn’t have been able to do that.
Adam Harstad (00:58:45):
And now you can.
Rob Collie (00:58:46):
Now we absolutely can. I loved that your arc so far in life goes
from some sort of genius to the niche where I know nothing. There's
something truly delicious about that and evolutionary.
Adam Harstad (00:59:02):
The fool thinks himself wise and the wise man knows he's a
pool.
Rob Collie (00:59:04):
Exactly. And you can never pronounce yourself cured of this, of
this disease because the moment you do now, you're back to the
trap. You can never pronounce yourself healthy in this regard. It
requires vigilance, constant vigilance and constant growth. But I
like to think that I've had a similar arc, I've been on a similar
arc from the know it all to know nothing and becoming comfortable
with that. Did you watch Ted Lasso?
Adam Harstad (00:59:31):
I've seen the first season, yeah.
Rob Collie (00:59:32):
The speech he gives about be curious in the bar right before he
kicks the guy's ass in darts?
Adam Harstad (00:59:38):
Yeah.
Rob Collie (00:59:38):
I found that to be one of my top 10 favorite moments of anything
I've seen on any screen, was that sequence. Be curious.
Adam Harstad (00:59:46):
The cool thing about being wrong, it sucks being wrong, there's
that whole ego threat where we identify ourselves with our
positions and it's hard to get past that, even somebody like me,
who's had so much experience being wrong. There's that ego threat
that I don't want to be wrong, but the cool thing about being
wrong, it's the first step to stopping, being wrong. You can't stop
being wrong until you realize you're wrong in the first place.
Rob Collie (01:00:09):
It is a very difficult thing to come to terms with. I'm much better
at it than I used to be. That's the only thing I'm comfortable
saying. Again, I got to be careful. You can't be the one that says,
"Yeah, I used to be like that, but I'm not like that anymore." "Yes
you are."
Adam Harstad (01:00:26):
It's like modesty. It's impossible to declare yourself modest.
That's right. I am probably the most modest person you've had on
this show. If I'm being honest here. Overwhelming modesty.
Rob Collie (01:00:38):
If I say modesty is not my strong suit, am I being modest?
Adam Harstad (01:00:42):
I can self-deprecate better than anyone. I'm the most
self-deprecating person you'll ever meet.
Rob Collie (01:00:50):
You know this from an emotional standpoint even from past years.
After one week of play, you still would want to draft very
differently based on what you saw.
Adam Harstad (01:01:00):
I think that's a trap though.
Rob Collie (01:01:01):
I agree.
Adam Harstad (01:01:01):
Yes. Emotionally people way overweight and their priors aren't
weighted enough. They're reacting too much to new evidence and
that's why I write that piece every year around week four and I
say, "It might surprise you to know that pre-season ADP at this
point still has every bit as much predictive power." Now, obviously
some blend between the two, a mix of pre-season ADP plus year to
date results will outperform either alone, but it's really a lesson
about not discarding what we thought we knew just because somebody
makes a huge, impressive catch on Monday night football.
Rob Collie (01:01:32):
Well, let me see if I can make a subtle, not a point, but a
question. There are situations where we can talk about this is our
best possible tool. It's 40% better than chance at doing something.
You'd still be better off betting, if you could bet the field, if
you could bet against the predictions, you'd still do better.
There's a weird, subtle distinction there between best possible
predictive power, which is factual, empirical and at the same time
betting that it was wrong. The ha-ha still has some less than truth
to it.
Adam Harstad (01:02:08):
Yeah. I'd say my entire niche in the industry and it's a weird
niche to occupy, but I think my entire niche is just I'm the guy
who knows nothing. I just aggressively embrace that uncertainty and
rather than trying to know more things, I feel like I try to
expound the limits of the things that I don't know. It really does
change the way that you play and you manage your teams. And one
example, one of my favorite things that I've done is about six
years back, I looked at how players age and the popular conception
of how players age is something called an age curve.
Adam Harstad (01:02:44):
You enter, you improve for three or four years, you hit your peak,
you maintain your peak for however many years, depends on your
position. And then you decline and then you're out of the league.
And if you plot all those points together, it looks like a curve.
And there's a lot of ways to drive this curve. If you average
performance at every player age, it'll make this beautiful curve.
And the age curve is an example of thinking that you know
something. If you look at a player, the age curve will say, this
player is going to do this this year, this next year, this next
year, this the year after that.
Adam Harstad (01:03:12):
He's going to be out of the league at this age. It represents
certainty and I never liked age curves because if you actually look
at NFL careers, very, very rarely do they take on a curve shape.
There's a lot of fluctuating, a lot of up and down. There's very
few players who I would look at their career and say, a curve
really describes what's happening here. And so, I gathered 30 years
worth of data about year to year fantasy football value. And I
said, "Let's try to make a mortality table out of this
instead."
Adam Harstad (01:03:42):
If you're familiar with life insurance, in life insurance, they're
basically making a bet for a life insurer that you will survive to
this age. You'll survive to that age, but it's not a curve. It's
not like you're 80% alive at age 64. You're still either a hundred
percent alive or you're a hundred percent dead. It's a very binary
state. They do tables that look at it probabilistically. And I
looked at NFL player careers. And so, one thing is if careers are
curve shaped, you would expect a player's last relevant season to
be worse than his second to last relevant season, on average.
Adam Harstad (01:04:13):
But I looked at the top hundred wide receivers and running backs of
the last 30 years and it was about 50, 50. 50% of them did better
in their last season, 50% of them did worse than their last season
before they just fell out of relevance completely. And so, I did, I
modeled and I found that I think the mortality table is actually a
pretty good fit for what's happening is players are basically
reaching their true level of play. They ascend, they learn the
position and then they reach their true level of play.
Adam Harstad (01:04:39):
And once they reach that level of play, they pretty much maintain
that level of play. They'll have up years, they'll have down years,
but they're fluctuating around that true level of play until at
some point and usually without any warning whatsoever, they just
fall off completely. They're great. They're great. They're great.
They're done. And when you start thinking about aging like that,
this wide receiver is 31, so I expect him to be a little bit worse
than he was last year, and it's more, this wide receiver is 31, so
I expect him probably to be about as good as last year or also
maybe he will be completely useless.
Rob Collie (01:05:11):
Love it.
Adam Harstad (01:05:12):
And the way you manage your risks is very different. When an age
curve would tell you that a 28-year-old wide receiver is a very
safe bet. He's still on the upslope of his curve. He's going to get
better or he's going to maintain, you have years before you have to
worry about him disappearing. And you look at Dez Bryant who is an
All-Pro wide receiver, never did anything after age 28. He was done
and an age curve would never predict that. Whereas a mortality
table would say, it's unlikely, but this is well within the range
of possibilities.
Adam Harstad (01:05:40):
I played dynasty leagues where you keep your players from year to
year, basically indefinitely, and it really makes you hyper aware
of the risks involved. You never enter a season thinking that
you're set at any position because you look and you're like, "Well,
there's a 20% chance that this guy's just done. There's a 10%
chance that this guy is just done." You add up all of those chances
across everyone. And a lot of guys on my roster are never going to
have another good season again.
Adam Harstad (01:06:06):
And on the flip side, sometimes you're more likely to take a chance
on old players because well, everybody thinks he's in the decline.
Julio Jones would be a great example this year. Everybody thinks
he's going to be worse this year, but maybe he's not. Maybe he's
just exactly as good as he has been for another five years. That's
in the range of possibilities. It's unlikely, but it's not
fundamentally unforeseeable outcome.
Rob Collie (01:06:31):
This tickles two ideas from daily life, whatever. One is, I have a
joke, which is the average has a population of zero. When you were
describing the curves, this is the aggregate average of everything.
All the players that have ever lived. That curve never describes
any one player.
Adam Harstad (01:06:52):
It's called the ecological fallacy, if you want a name for it.
Rob Collie (01:06:58):
Ecological fallacy. I think you'll appreciate this. From the
business side for a very, very, very long time, honestly, even to
this day, really, this still persists a lot of companies, they have
their top number. They have their top level number for whatever
metric it is, profit margin or customer retention or whatever. They
know what their all up number for the entire company is, but they
lack the ability to decompose it into the various segments of their
business. If they've got 50 salespeople out there, what's the
average win rate for a deal?
Rob Collie (01:07:28):
Now, let's say it's 25%. But if you go look at the 50 salespeople,
none of them are at 25%. You've got some that are down at 3%, and
then you've got a handful of outliers at 45% and you're not
learning anything. There's no way to improve. One of our core
philosophies is you need to know your top level number. You
absolutely do. You need to know what your final score is, but the
game is improvement. It's not knowing.
Rob Collie (01:07:54):
Knowing is a necessary step for improvement and you cannot improve
unless you're able to effortlessly subsegment and see what's really
going on to focus your attention in the right places. There's
different prescriptions for different places. And so, when you're
talking about this curve versus mortality risk, I was lighting up
like, "Yeah." And the other thing that I've been wrestling with and
really just torturing some of my friends with for a long time is
this notion of single trial experiments and probabilistic models
applied to single trials.
Rob Collie (01:08:27):
Julio Jones isn't going to have an 80% season probably. He's not
80% relevant or 80% still in the league. He's either going to have
a good season probably or vanish from relevance. Probabilistic
models of who wins the election. I know that's our best tool for
predicting who's going to win an election, but there's something
fundamentally silly about saying that someone has an 85% chance to
win the election. They're not going to 85% win ever. They're going
to win or they're going to lose.
Rob Collie (01:08:56):
And so, we need to come to terms with that, so when it's wrong,
when something like that is wrong, it's more wrong than 15% wrong,
in my opinion. Anyway, we don't need to pick that fight. I've done
that a million times, a million different people.
Adam Harstad (01:09:11):
Nassim Taleb versus Nate Silver debate.
Rob Collie (01:09:13):
Yeah. I Think I'm on the Nassim Taleb side of that debate.
Adam Harstad (01:09:16):
I tend to be more with Nate Silver, where on any given prediction,
you can't know if it was a good or bad prediction, but over a large
enough sample, you can test your calibration. And if the things
that you say will happen, 15% of the time are happening 15% of the
time, I don't think it says anything about the predictions
themselves, but I think it says something about you as a predictor
being fairly well calibrated.
Rob Collie (01:09:38):
Sure, but of course, all that matters to me is its accuracy and of
course that includes both the predictor and the methodology.
Adam Harstad (01:09:45):
Well, here's an interesting thought experiment. Let's say we have
two different models. We'll go back to Ja'Marr Chase, rookie wide
receiver for the Bengals, and we have two different models and
these models are both extremely well calibrated. And you feed them
different inputs and they will tell you what percent chance a
player has of becoming a Hall of Famer. And we've tested these
models, and we know, let's just say by faith that these models
we're going to stipulate that they're extraordinarily well
calibrated.
Adam Harstad (01:10:11):
And one model says that Ja'Marr Chase has a 15% chance of becoming
a Hall of Fame wide receiver, and another model that he has a 40%
chance of becoming a Hall of Fame wide receiver, because it's
looking at different inputs and those inputs predict different
chances. You know what are Ja'Marr Chase's true chances of becoming
a Hall of Fame wide receiver? Is it 15%? Is it 40%? Is it somewhere
between the two? If he does become, how right or wrong was each
prediction? If he doesn't become, how right or wrong was each
prediction?
Adam Harstad (01:10:40):
Two predictions, those can both be right. It seems fundamentally in
conflict that one person says he has a 15% chance, one person says
he has a 40% chance and both can be right at the same time, and yet
it is. We're stipulating upfront that both are right. And I think
when you grapple with data and especially predictive data, you need
to develop a certain comfort level with that ambiguity, with we're
just never going to know. We never will know. And there's no way we
possibly could know, because as you say, it's a single run trial.
We can't rerun it.
Adam Harstad (01:11:12):
But if we're aggressively testing our calibration and we're working
hard and making sure that the things that we're saying are
correlating with reality to some extent or another, it's almost
like you're not really having faith in the predictor so much as
you're having faith in the process to produce results that will
have meaning and add value.
Rob Collie (01:11:32):
The most sophisticated version of my objection to all of this is
actually, and not to what you were saying. I completely agree with
the things you were saying. The most refined version of my
objection to Nate Silver is that I think the domain he's applying
these methods to is exactly the wrong one to give these methods a
positive reputation, because here's the thing, on the eve of the
election, the universe, this is can be a little weird. We're going
to have to get a little weird to explain it, but the universe
actually knows who's going to win.
Rob Collie (01:12:03):
There's no chance that it's going to swing 85, 15% tomorrow. That's
not going to happen. We already know. The universe knows. Whereas
in the case of Ja'Marr Chase, I can make this fuzzy argument that
even the universe doesn't know if Ja'Marr Chase is going to be
successful.
Adam Harstad (01:12:18):
I don't know if I took that argument. I have a thread on this on
Twitter. That's saying basically what you’re saying. If I flip a
coin and I say, "What are the odds that this coin comes up heads?"
First, you want to know if the coin is biased, assume it's a fair
coin. It comes up heads 50% of the time. I flip a coin. I say,
"What are the odds this comes up heads?" You're going to tell me
50-50.
Adam Harstad (01:12:36):
If I were flipping this coin in front of some alien
superintelligence who was so innately good at physics that he could
divine from the starting point and the forces involved and wind
resistance, he could tell with absolute certainty, which side was
going to come up. He'd either say a hundred percent heads or 0%
tails. And again, both of you are right. You say 50-50, you're
correct. The alien says a hundred percent or 0%. If he's well
calibrated, he's also correct. Fundamentally odds are not measuring
anything intrinsic to the event itself.
Adam Harstad (01:13:06):
Odds are a measure of our ignorance surrounding the event itself.
And people will get different odds for different things because
they have different levels of ignorance. If I hold up a card from a
deck facing me and I say, "What are the chances this card is an
ace?" You're going to give me one number. If I then show you 13
cards from the deck and say, "What are the chances these cards are
an ace?" You're going to give me a different number. The card
itself hasn't changed.
Rob Collie (01:13:29):
I know.
Adam Harstad (01:13:30):
The odds are really meaningless to the card itself. They're just a
measure of your level of knowledge and what you know. The odds are
saying more about you than they're saying about the card.
Rob Collie (01:13:39):
Well, let me push back a little bit. Let's take your alien
intelligence coin flip example. And instead, have it be a billiard
ball on a frictionless surface or damn near frictionless surface,
so that we could make the billiard ball bounce off of the rails 40
or 50 times. Once you get that number of collisions, maybe 50 isn't
sufficient, maybe it takes a hundred.
Rob Collie (01:14:01):
Once you get the number of collisions high enough, even the alien
intelligence won't be able to predict where it ends up at the end
of a hundred bounces because even the quantum uncertainty and the
vibration of the ball magnifies to great enough difference that you
don't end up in the same place.
Adam Harstad (01:14:15):
Well, but the quantum uncertainty, that's what I'm saying. Again,
the odds are a measure of our own uncertainty.
Rob Collie (01:14:20):
Completely.
Adam Harstad (01:14:20):
So if you increase the alien's uncertainty, you're going to
decrease the odds [inaudible 01:14:24].
Rob Collie (01:14:24):
Okay. But I'm saying, the distinction between the two. No amount of
tomorrow uncertainty is going to change the outcome of the
election. All that uncertainty balances itself out, large numbers.
Whoever wins the election was going to win the election in a
million sub universes or a billion sub... Whatever. They're going
to win on all of them. Whereas Ja'Marr Chase, I think even the
universe with perfect knowledge, you don't actually know. There's
too many branching points in the equation of whether he turns
into...
Rob Collie (01:14:52):
And this is why I think intuitively the masses have a point when
they make fun of Nate Silver getting it wrong. Whereas they would
be more accepting of the Ja'Marr Chase Hall of Fame percentage. We
understand it's uncertain, but they get that there was a more
wrongness about the election prediction than there is about the
Ja'Marr Chase prediction.
Adam Harstad (01:15:15):
Yeah. I think the famous quote is, "All models are wrong, some are
useful." And that's the question, any model. What you're getting
at, I think is, there's something called epistemic uncertainty and
something called aleatory uncertainty. I know you like learning
new-
Rob Collie (01:15:31):
[crosstalk 01:15:31]. Oh my God. Write these down, Luke, I need
to...
Adam Harstad (01:15:35):
Epistemic uncertainty is things that are theoretically knowable, we
just don't know them. And I think you're suggesting that an
election outcome, the day before that's epistemic uncertainty. This
is fundamentally knowable. We just don't know it. And the odds that
we're assigning are representing that epistemic uncertainty and
aleatory uncertainty is irreducible uncertainty. It's even if we
knew absolutely everything, this is fundamentally unknowable, it's
irreducible.
Adam Harstad (01:16:03):
And it seems to me that you're suggesting that probabilistic
forecasts in the case of aleatory uncertainty are okay. Something
like Ja'Marr Chase is going to make the Hall of Fame, he won't make
the Hall of Fame. Here's this chance. Whereas in the realm of
epistemic uncertainty, you're saying there's just something hinky
about them. There's something fundamentally weird
uncomfortable.
Rob Collie (01:16:23):
It's not that they're not okay. It's just that they're a little
less okay.
Adam Harstad (01:16:27):
It's fundamentally different.
Rob Collie (01:16:28):
An 85% number about something that is only going to be zero or a
hundred, and I believe we'll be the same zero or a hundred, no
matter what happens in between now and then, there's something very
funky about even bothering to put a number. It's like the illusion
of confidence. It's the illusion of precision. 85.6%. Not 0.5%,
0.6%, Nate? It's like what you were saying earlier like, when
you're deep into analytics, you also have to maintain that other
wisdom.
Rob Collie (01:16:59):
I just think Nate Silver in particular, this forecasting, the
result of a national election, it's just the worst possible poster
child for the analytics industry. We're drawn to it. We look at it
like crazy, but just the worst place to have our reputations as
professionals displayed.
Adam Harstad (01:17:16):
Yeah. It's funny. Footballguys, we obviously the people who write
for Footballguys who are big into fantasy football we're pretty
fluent with data and probabilities because what you're saying about
your BVBA, that's basically data and probabilities. That's what
fantasy football is. And we had a really interesting discussion
after the 2016 election where some people were saying Nate Silver
was very, very wrong because he said there was only a 36% chance.
And other people were saying everybody else had the chances at 1%
to 10%.
Adam Harstad (01:17:45):
Nate Silver was the only one out there saying there's a very real
chance. 36% might not sound that much, but it's about the odds of a
kicker missing a 48-yard field goal. If your team is lining up to
kick a go ahead field goal from 48 yards with time expiring, how
comfortable are you? That's what 36% feels like. It happens a lot,
36% of the time in fact. It's interesting that even in a place
that's so comfortable dealing with risk and probability, there's
still that fundamental divide and I think it really boils down to
the appropriateness of putting numbers on epistemic versus aleatory
uncertainty.
Rob Collie (01:18:20):
And I was with you for a long time that there's this over
precision, but Phil Tetlock, I don't know if you're familiar with
him, works on a project called The Good Judgment Project. And he
wrote a book called Super Forecasting. And basically the CIA had
this forecasting competition. They have a hundred questions. Here
is this Middle Eastern country. What are the chances that they're
going to suffer a coup in the next seven years? And Tetlock
basically his approach to it was this wisdom of the crowd approach.
A modified wisdom of the crowd.
Rob Collie (01:18:49):
Is he had a bunch of people making predictions, and then over time
he saw that some people were more consistently right and so he
started weighting their predictions more and more. And eventually
he came up with this group of people who were definitely out
predicting everybody else, usually with no specific domain
expertise. And he named them super forecasters and then he's done a
lot of studies about what makes these people so good at predicting
basically one off events. And he found that when you look at the
precision, the precision is meaningful.
Rob Collie (01:19:17):
If a super forecaster says there's an 83% chance that there's going
to be a coup in this country versus an 80% chance, instead of just
rounding to the nearest 10, he finds that that 3% is meaningful.
That ignoring that 3% makes the aggregation worse, makes the
outcomes less predictive. And so, I would say that for the most
part, I'm skeptical of that appearance of certainty and that
appearance of precision, just because most people are not super
forecasters, but I have become more open to the possibility that
actually no, that 0.3% might be doing some real work there.
Adam Harstad (01:19:50):
Yeah. I think the distinction again, when I'm not being just
bombastic about it, when I'm really clear about it. My only
objection here really is the reputational risk inherent.
Rob Collie (01:20:03):
Which is fair.
Adam Harstad (01:20:04):
I do want, if we're going to be making predictions, of course I
want the methodology to be rigorous, if you're not artificial about
it, the methodology that produces more precise numbers probably has
more internal checks in the first place. It probably is the process
that you use to arrive at that number was probably a superior
process to the one that arrives at the round number. I'm okay with
the methodology part of it. It's just that when we put it out in
the public, it's the old nerds versus jerks argument all over
again.
Adam Harstad (01:20:34):
But now we're in our middle age and the whole notion of using data
to do things gets cheapened every time we put an 85% number on an
election, and then it's either a much closer than expected or it
goes the other way. I just wish again, for reputational reasons
only, it's the rear view mirror, not the forward only allowed three
predictions for an election. This person wins, that person wins,
too close to call.
Tom Larock (01:21:00):
I just want to say, Rob, I'm not going to disagree with anything
you've said, but what you're missing is the fact that 86% of
Americans don't understand what the P value really means. Your talk
about, this is a bad thing to put in front of people, but the real
thing is, is that Nate Silver goes out and he describes things as
if people understand what these concepts are and they really
don't.
Rob Collie (01:21:29):
I'm saying two things. One is that you go to war with the army
you've got right. The population that you've got. And so, they
don't understand P value is great. But the other thing I'm saying
is that there's something also fundamentally meaningless about his
P value. It's because of this different aleatory versus
pistemological, was that it?
Tom Larock (01:21:46):
Episcopalian, I thought.
Rob Collie (01:21:48):
[crosstalk 01:21:48].
Tom Larock (01:21:48):
Episcopalian
Rob Collie (01:21:52):
Epistemic.
Adam Harstad (01:21:54):
If I can have one rebuttal.
Rob Collie (01:21:56):
Of course, we'll give you the last word.
Adam Harstad (01:21:58):
I'm very, very respectful of arguments that it's not enough to be
right, it's are you advancing the cause? I think trying to think
tactically, that's a very important skill to have, but I would also
say, I don't think that there's been anybody in America who has
done more for the understanding and acceptance of analytics, than
Nate Silver over the last decade. I think the public is much more
open to conversant in the language of analytics and I think that
stems primarily to Nate Silver in FiveThirtyEight in his election
forecasting.
Adam Harstad (01:22:32):
Maybe the 2016 election was a setback, but I think on the whole,
he's been a very positive, because I see it in fantasy football
too. Just the acceptance of alternate data sources. 15 years ago,
you mentioned Football Outsiders and people dismiss you out of
hand. Nowadays you get random people off the street who are citing
DVOA and EPA per play to you. And I think Nate Silver was a big
part of that revolution.
Rob Collie (01:22:56):
I agree with you. I think that's true. Can I not disagree, but
adjust it a little bit? Let's bring back the average has a
population of zero. I think he's probably reeled in some percentage
of the population to the idea of analytics while pushing another
percentage further away.
Adam Harstad (01:23:15):
Maybe.
Rob Collie (01:23:15):
But then in terms of real numbers, we're seeing, I totally agree
with you. The analytics has benefited as a field from the exposure
he's brought to it, for sure. But even if that were, and again,
we're just making up numbers at this point, even if that were 20%
of the world that was now more receptive as a result, that would be
massive. But I see a lot of ha-ha out in the world and I just don't
want the methodology that he uses to be unfairly judged by the
domain in which we find it most entertaining.
Rob Collie (01:23:48):
And I do think that the reason why he's so famous for the politics
stuff is more for the entertainment reason, which is not the right
reason. I don't have any objection to the guy or his methods or
anything like that. It really is just like, "Ugh, I wish we could
be fascinated with something else."
Adam Harstad (01:24:06):
Maybe.
Rob Collie (01:24:06):
Adam, man, a thank you is appropriate because I've enjoyed the hell
out of this.
Adam Harstad (01:24:10):
I did too. It was a lot of fun.
Announcer (01:24:11):
Thanks for listening to the Raw Data By P3 Adaptive podcast. Let
the experts at P3 Adaptive help your business. Just go to
p3adaptive.com. Have a data day.