Oct 27, 2020
Brad Miller and Kai Hankinson from Agree Media were Rob's first clients at P3 before it was even called P3! These guys have been close friends and business partners for a long time, and they know a lot about data. We cover:
Episode Transcript:
Rob Collie (00:00:00):
Hello again everyone, one of the gifts of my career is all the
interesting places it's taken me during some interesting times and
all the interesting people I've met along the way. Well, today's
guests, Brad Miller and Kai Hankinson are two of the most
interesting people I know. Ivy League MBAs, entrepreneurs, rabid
users of Power BI. Along the way, they've written poker bots, lived
in India, they've been sued by Facebook, and they've lived off the
land in Hawaii. But they're more than just interesting people to
me. I'm really grateful to say that they're also my friends.
Friends who've been along for the whole ride. They were there
before P3 was even a company. And really they were the guinea pigs
for the whole idea. When I had a full-time startup CTO job, they'd
hire me to consult with them on the side, even though I wasn't
taking consulting terribly seriously yet.
Rob Collie (00:00:49):
So they watched P3 develop from literally zero into the nationwide
firm we are today. More than watched actually, they helped and vice
versa. I've been along for their journey as well. So what I hope
you hear is an authentic conversation between friends. A
conversation revolving around data, for sure. But again, that human
element looms large, the filter was completely off for this one. We
talk a lot about failure, rebounding from it, learning from it. We
laugh a lot about how hard it was to get them onto the Power BI
train. Even though today, they can't live without it. And I think
you'll see there's no shortage of personality in these two. It's a
long one, but I don't think there was a single dull moment in the
conversation. So let's get after it.
Announcer (00:01:33):
Ladies and gentlemen, may I have your attention please?
Announcer (00:01:37):
This is the Raw Data by P3 podcast with your host, Rob Collie. Find
out what the experts at P3 can do for your business. Go to
powerpivotpro.com. Raw Data by P3 is data with the human
element.
Rob Collie (00:01:53):
All right. Welcome. My California startup entrepreneur friends.
Thanks both of you, Brad and Kai for coming on. Really been looking
forward to this. You guys have been in business together for how
long now? How long has Hankin Miller Enterprises been a thing?
Brad Miller (00:02:10):
I think we got started what, 2005, Kai?
Kai Hankinson (00:02:14):
I think we got started in 2004 actually.
Brad Miller (00:02:16):
2004, so it's a long marriage.
Rob Collie (00:02:20):
Yeah, I was going to say, and one of you has already forgotten the
anniversary. One of you's already forgotten how long. So it's a
realistic marriage, isn't it?
Brad Miller (00:02:30):
It's a very realistic marriage on a lot of fronts.
Rob Collie (00:02:33):
Yes it is. So you guys met in business school, right?
Kai Hankinson (00:02:37):
Yep.
Rob Collie (00:02:37):
And not just business school, right? Ivy League business school.
That's where you guys met. I'm an engineer sort of originally like
that sort of what the machine churned me out as, as a 22 year old.
From my perspective, you guys were in a completely different world.
You were in the essentially like the financial and business world
with almost like this made man sort of circle, in a way. Right? But
today I think of you as data professionals in every bit as much of
the real sense as I am. So Wall Street, private equity, my
expectation as a borderline Vulcan nerd is that once you're in that
world, you would just stay there. You know, you would just stay in
that world. You would stay circulating in that Wall Street and
private equity world. You'd have other people doing your data for
you. You wouldn't be getting your hands dirty in that. You'd just
be too important for it. That's my expectation as an outsider, but
that's not where you find yourselves.
Rob Collie (00:03:39):
And so I just want to start here, why did you leave? Why did you
leave that world to go do your own thing? And some of the things
you've done, as we're going to get to, are very interesting. What
led you to branch out like that?
Brad Miller (00:03:52):
Kai, you want to take that one?
Kai Hankinson (00:03:54):
Yeah. Brad and I, we met at the first day at Columbia Business
School, and we were assigned this little group of four students
together that did a semester long project. Brad and I hit it off as
friends, and it wasn't long before we started up a weekly poker
game at my house where we would take money off of the rest of our
business school friends, who mostly came over to socialize and
drink the free beer we bought them. We were taking the poker much
more seriously just because that was our thing, and we wanted to
win.
Rob Collie (00:04:29):
I've had friends like you at poker games. You ruin everything.
Kai Hankinson (00:04:37):
Yeah. So we had a lot of fun playing poker through business school.
It was around the time that the Rounders movie came out. No, we
went to school 99 to 2001. So we were in Manhattan and we'd
occasionally hit up those Rounders style clubs in Manhattan that
the movie was kind of written after. Those places used to scare the
shit out of us, in a good way. But we would just walk into places,
scared to death, just these little business school dorks. Trying to
look like we knew what we were doing, and we pushed some chips out
there.
Rob Collie (00:05:09):
And then Teddy KGB.
Kai Hankinson (00:05:11):
Yeah.
Rob Collie (00:05:11):
Sitting across the table. Yeah.
Kai Hankinson (00:05:14):
Yeah. You don't want to mess with Teddy KGB. Anyways, we graduated
business school finally and went to for the man. I was on Wall
Street for Merrill Lynch and Brad went out to California with
Deloitte, but we just kept in contact about the poker because we
were really interested in it. We started studying the artificial
intelligence projects going on in poker at some of the institutions
like University of Toronto, I think was a big one. This was back in
the day when the AI had beaten the chess problem, but hadn't yet
really made a dent in the poker problem because it was a tougher
problem. Chess is a perfect information game. Everyone can see all
of the board and poker isn't, and because of that, you get the
whole component of bluffing and everything else.
Kai Hankinson (00:06:04):
And so it's just a harder problem for AI, but we were really
interested in their attempts to solve it with AI. We ended up
getting this idea that we could break the poker hand up into the
parts that were best for AI and the parts that were best for human,
and just divide up the workload that way. And together build a best
case player. We ended up raising some money. We actually raised the
money in 2004, which was the reason for my answer, and we quit our
jobs in 2005, which was a reason for Brad's answer. We raised the
money and left our gigs, and started up a company dedicated to
playing online poker with artificial intelligence assistance.
Brad Miller (00:06:51):
I will say that it was easier for me to leave my job than it was
for Kai to leave his job. Going back to your question, I had always
considered myself an entrepreneur and I knew that's what I wanted
to do. So my time in management consulting was time I was putting
in there. It's also very difficult to get rich charging by the
hour. Right? Which is the consulting model. So I knew wasn't really
my long term goal. Kai, on the other hand, had landed a very plush
gig, trading equity derivatives for Merrill Lynch. It took me a
while to convince him to leave. I think, at the time, his wife, at
the time, wanted to move back to California, and the New York
lifestyle might have been killing him a little bit, and so that
kind of helped the cause.
Brad Miller (00:07:41):
During the whole time I had been playing on the nascent online
poker rooms, and there was some really basic software that was
available at the time, I forget the name. There were all these
upgrades that I wanted to make to the software that I was like,
"the guy's missing this, he's missing that, he's missing this." And
so I'd keep telling Kai about it, and Kai would agree and be like,
"what you're describing is a lot like kind of the software and the
UI that I use at Merrill to trade derivatives." And so we started
to design what this ultimate software product might look like?
Rob Collie (00:08:20):
Did you hear how Brad started that? He said, "it's really difficult
to get rich charging by the hour." You had to throw that in there,
Brad, you had to cut me. As the owner of a consulting firm, don't I
know it? You know?
Brad Miller (00:08:33):
I think the difference is you're the owner, so if you've taken that
step and taken the additional risk, right?
Rob Collie (00:08:40):
That's true. I'm still aspiring to be you guys when I grow up, and
you're still aspiring to be someone else when you grow up. It's how
the game is played. If I can paraphrase the last couple minutes, we
really enjoyed playing poker with our business school friends that
weren't as good at it, and taking their money. And then we said,
what if we could take that and really scale it? What if we could do
that online? That'd be great.
Brad Miller (00:09:07):
I think that's right. One more thing I'd want to throw in though is
that I think both of us live by a kind of philosophy where if
you're not willing to publicly admit that you do something, then
you probably shouldn't do it. And so it's kind of a motto we live
by. I know I definitely felt like I was living two lives at the
time. I was living Brad, work Brad, for a consulting firm. And then
I was living private Brad, and I just really didn't enjoy that. I
was like, I want to be one person, I don't want to have to put on
an act, I don't want to have to satisfy the man.
Brad Miller (00:09:49):
Things like Kai went to this mushroom ceremony a couple months ago
and had an amazing experience, and wrote an email to our entire
team about it, explaining the ceremony and what he had learned. You
can't do that at Deloitte. If you want to go run around naked at
Burning Man, you should be able to go run around naked at Burning
Man; but Deloitte customers probably wouldn't appreciate it. So,
that's what I mean by kind of two separate lives.
Rob Collie (00:10:17):
There's all these like workplace personality assessment, there's a
million companies that do this. The ones that I've done, I've done
the same one a couple times at different points in my life. They
give you two pictures. They give you one of yourself in the
professional environment, and one of yourself in the personal
environment. There's almost also this distance score between the
two.
Rob Collie (00:10:41):
The mediator in the first session, when I took did this at
Microsoft, told us that those of you with a large distance between
these two, the most authentic moment he could offer, he was just
going through script for all day. Right? But this one moment you
could sort of tell he was telling you something that was real. He
was sort of dropping the script and he was saying, "if you have a
really large distance there, it takes its toll." He said it that
slowly, and then he repeated it, "it takes its toll." I'm sitting
there looking at my sheet, and me and one other guy and this room
of like a hundred people, like we're these outliers with this most
dramatic difference between our personal and our professional. And
I'm just like, "Ahh!" You know? Took it again recently, but my
distance isn't as great. Both sides of me has changed over the 10
years intervening. They've sort of moved towards each other, and I
feel a lot better.
Brad Miller (00:11:35):
Yeah.
Rob Collie (00:11:35):
I can certainly appreciate that. Did you get rich with the poker
thing?
Brad Miller (00:11:39):
So scale, we did. We ended up moving the company to India because
we wanted to use one of the outsourcing firms there. We were
working with one of the outsourcing firms. We ended up putting
together a group of a couple of hundred Indians that would use our
software that we had designed and developed to play online poker at
the various rooms. It was pretty crazy. We were doing things where
the software was recording every hand online at these poker rooms.
So we had data on every handle that was playing. So that was
accessible while you were playing against that opponent. Our
software allowed you to play upwards of 10 to 20 hands
simultaneously. We then started doing things, because it was all
through our UI, we would break the hand down into its various
components. For example, we would have an expert that would only
see hands that had pocket pairs in early position.
Brad Miller (00:12:45):
Whenever we got dealt up a pocket pair in early position, we would
route that to Indian call center agent number 12. So, on his screen
at any point in time, you could have 12 hands, only from the flop
forward, only pocket pie pocket pairs in early position. That's all
he knew. He was an expert. That's what he did. So we just designed
the whole thing kind of similar to how he would design a call
center, and how you would route different phone calls. We would
route poker hands the same way.
Kai Hankinson (00:13:19):
Yeah, in order to do that, guys and girls would get thrown into the
middle of poker hands. So in order to do that, and also to
facilitate the ability to play so many hands at the same time, Brad
invented this graphical representation of a poker hand to where, in
a quick instant, you could catch up on all the action that had
occurred up to that point. It really was one of the coolest things
we came up with, I think. It was just so powerful to be able to,
even if you were playing in the hand the whole time and you forget
that a lot of the little stuff, and with this, you can at any
point, look back at every little bit and be like, "oh, that's
right, he did call that cold on the flop."
Rob Collie (00:14:02):
Do you have a screenshot of that laying around anywhere? This is a
data visualization technique, right? It'd be a really interesting
novel thing. We won't actually do this, but it'd be a neat thing to
build the Power BI custom visual that reimplemented what you
designed. If we were just laying around with nothing to do, we
would absolutely do that.
Brad Miller (00:14:22):
Yeah. It'd be cool. I'm sure we have it somewhere. It's also just
such a neat experience because they don't play poker in India. And
so we had to train everyone up from the get go. We also couldn't
let them that they were actually playing real poker because they
were betting, in some cases, hundreds of dollars per hand, they
were only making $10 a day. If they knew that they had hundreds of
dollars on the line in any given hand, they would just freeze and
wouldn't play optimally. So we basically had to tell them that they
were just testing the software and they were trying to train the
software up. So it was a crazy experience.
Kai Hankinson (00:15:03):
That was a wild experience. You would walk around that floor, that
call center floor, overlooking the shoulder of all these little
kids. They were like 20, 21 to 24 years old, just so innocent and
fun. You'd look over their shoulder and you would know what was
going on, and I'd be nervous as crap like, "oh please, please don't
screw this one up. You got to raise him here, you know, right?" I'm
thinking, but I don't want to interrupt him because he is also
playing four hands at the same time. What a wild experience.
Rob Collie (00:15:40):
Now, Luke is usually just the silent producer on these things, but
in the same timeframe, Luke was playing a lot of online poker.
Luke, now do you see what you were up against?
Luke (00:15:51):
Well, yeah, I might have played against you, and that's the reason
why I sucked so bad.
Brad Miller (00:15:58):
Now, we were playing primarily at the lower limits in the limit
games. So, once you get to the high limit stuff, the software
wasn't able to compete. So we were mostly in it for the quantity of
the quality.
Kai Hankinson (00:16:14):
We were playing quarter million hands a day, quarter million hands
of poker a day at the end. So, maybe we should say though that
eventually we got with the Indian operation to where we were highly
poker profitable, but the company wasn't yet profitable. We weren't
covering our full operating expenses. We were running out of our
runway from the money we had raised. We decided to make a bet that
we could automate the rest of the hand, and both cut down our cost
structure quite a bit and play even better poker than we were with
the human training.
Kai Hankinson (00:16:52):
We eventually packed up from India. We came back to San Diego,
boarded ourselves up in an office, and set out on a six month
endeavor before we ran out of money to fully automate the hand. We
made it just in time, at the end, we got up to a quarter million
hands a day, when it was just servers, no operational nightmares,
no humans to maintain and get everybody aligned up, logged in, just
servers, just pounding out hands on every table we could get a seat
on online. You say we were primarily lower limit, I think the truth
is that's only because that's primarily what's played. We had guys
at every limit poker seat. It was just that there's very few high
limit tables.
Rob Collie (00:17:38):
Did you ever suspect that you weren't alone in this? That there
might be other similar operations going on, and some of your hands
you were playing, you were also playing against other bots?
Kai Hankinson (00:17:50):
You definitely wondered. It's just so unknowable.
Rob Collie (00:17:55):
Yeah.
Kai Hankinson (00:17:55):
People would accuse us of being bots regularly. I think that's just
an accusation that probably flies around online quite a bit. We had
a whole setup where when one of our players was mentioned, it would
get routed to a human whose job was to like pipe in on the comment
thread and come back over the top, "your mother is a fucking
bot."
Rob Collie (00:18:20):
You had a specialist for that as well.
Kai Hankinson (00:18:25):
Yeah. Yeah, we did. We didn't like being called bots.
Rob Collie (00:18:28):
Yeah, those accusations that are true, those are the ones that hurt
the most. You know?
Kai Hankinson (00:18:34):
Yeah, we have a poker reputation to maintain as a big bluffer.
Rob Collie (00:18:41):
At the same period of time when you guys were doing this, I was
working at Microsoft and playing a lot of World of Warcraft
because, like you were saying earlier, you get this difference
between your personal and professional self. There's a point at
which you don't really pour as much into the Microsoft job, right?
You start staying up late playing these games. There's bots all
over those games. Those games are loaded with bots that are running
around farming in-game currency to turn around and sell it for real
world currency. You can report the bots, and if someone tries to
talk to them and if the bot seems human-like in its response, the
bot gets to go on its way. Look at the sophistication of the
CAPTCHAs that we have to navigate on the internet these days. We're
reaching the point where the CAPTCHAs are harder for us than they
are for the bots. We're not too far away from the point where the
bots are better at solving the CAPTCHAs than humans. Then what do
we do? It's sort of game over.
Kai Hankinson (00:19:42):
The best defense is if you can pass a quick Turing Test.
Rob Collie (00:19:46):
Oh yeah, of course, everyone knows about that. The Turing Test, if
it turns out that, in those situations, the bots are just routing
to a real human to pass the test, what are you going to do? Game
over. I asked, "Did you get rich playing with the poker business?"
You said, "Well, we scaled."
Brad Miller (00:20:08):
We did not, we got very poor with the poker business. There were
two things that went against us at the poker business. One was in
around 2008, they passed some legislation that basically killed the
industry for a while. All the poker sites were based in offshore
jurisdictions that the US government couldn't touch. So what they
did was they regulated the bank transactions between the US banks
and the poker sites. So there was no way to get money in and out of
the poker sites anymore. Which effectively killed it. Which was a
pretty smart way to kill it, if you're going to do anything. The
other one was we thought that the poker software would kind of go
the way of the market trading software, the stock market trading
software, the kind of stuff that that Kai was using at Merrill
Lynch. We thought that if you could build better software, great,
you deserve to win just like in the market. If you can build better
software, you're a hedge fund, you can build better software,
great. The poker sites don't like that. We didn't quite realize
that at first. The reason they don't like that is because what
happens is you take it to the extreme.
Brad Miller (00:21:20):
Let's say this one guy at the table and he kills everyone else.
Basically all the money goes to him. He stands up and he walks
away, right? The poker site doesn't make any money. If everyone at
the table is of kind of equal capability, then what happens is they
just push money around hour after hour. Everyone's just sitting at
the table, money's going back and forth. And the poker site takes
their percentage out of every hand or out of every hour or whatever
it is. So the poker site has a lot of incentive to make sure that
everyone is kind of at an equal playing field, an equal level. And
so they were very anti the kind of software that we were building.
So we had those two strikes against us with were too big to
overcome.
Kai Hankinson (00:22:10):
We were making about $5,000 a day in pure poker profit, and we were
on our way to getting rich. We just didn't get the money out.
Rob Collie (00:22:21):
So you got poor instead of rich playing poker. Which, when you say
it that way, makes it sound like it was predestined. I'm sure you
both took that really well. Right? How many years did you have sunk
in it at that point?
Brad Miller (00:22:37):
Well, that was about 2008, 2009 around there. So...
Rob Collie (00:22:41):
Wow. That's longer than I realized you guys were at it for four
years.
Brad Miller (00:22:46):
Yeah, roughly four years.
Kai Hankinson (00:22:48):
Yeah. Yeah. The hardest part for me was we had raised all our money
for that endeavor had come from friends and family.
Rob Collie (00:22:56):
Yeah.
Kai Hankinson (00:22:56):
So we were a total loss for our investors. That's one thing that I
love about our partnership, Brad and mine, is that we've been
through thick and thin together. You know? That's not the only
company that we had a bad outcome on. We're on our fifth startup
together, and we've had some depressing bankruptcies and some
exciting exits, and we've been through it all. I've seen a lot of
partnerships break up on the downside. I've seen a lot of
partnerships break up on the upside, even. When, all of a sudden
when you win and you exit, now there's something to fight over, and
partnerships break up. We weathered all that together, we're on
our, onto our fifth company together right now.
Rob Collie (00:23:36):
Yeah. That's amazing. Now, just a random side question. What do you
know about organic farms in Hawaii?
Kai Hankinson (00:23:46):
Well, after losing all my friends and family's money, I got pretty
down on myself, and Brad got right back on the horse and went right
into private equity and was off to work the next day. I took a year
and a half off of work just to kind of regroup and find myself. I
ended up during that year and a half over in Hawaii doing something
called WWOOFing. Which is working for free on organic farms. That
was a great time in my life. I did a lot of meditating, lot of soul
searching.
Rob Collie (00:24:21):
I now know too WWOOFers. Paul, at our company was telling me a
similar story one day about, "Well, you know, there's this time
where I ended up, you know, over in Hawaii and like, you know,
living off the land on this guy's farm that he let us..." And I'm
like, "Whoa, whoa, wait a second. I've I have heard this story
before." What are the odds? I'm a WWOOFer magnet, apparently.
Brad Miller (00:24:46):
I decided to buy a one way ticket to Hawaii. Bought one of those
kind of load of the ground trikes. I don't know what you call them,
but...
Kai Hankinson (00:24:55):
Recumbent trike.
Brad Miller (00:24:56):
Recumbent trike. Folded it up, put it in a baggage claim, and then
was sitting in baggage claim in Hawaii...
Kai Hankinson (00:25:03):
Baggage claim. And then, it was sitting in baggage claim in Hawaii,
putting it together. And then as soon as he put it together, he
just hit the road and stopped where he stopped.
Brad Miller (00:25:11):
Lived out of a tent. I went months without spending money. It's
crazy over there. You can just live off the land, there's avocados
and nuts growing everywhere.
Rob Collie (00:25:22):
I really need to try that someday. You're making it sound so
good... All right. So poker, Hawaii, private equity and you did
something else. What was the next one? Now we're really starting
to... AI and ML definitely qualifies as a data thing. There
would've been no real reason for our paths to intersect, especially
back around 2010. So what were you guys doing in 2010 when we
met?
Brad Miller (00:25:57):
At that time, we had gotten back together. I had come out of my
hiatus and at the time when we first met, we were running a
Facebook app called Page Rage. It exploded in popularity, it was
back in the Facebook app days. That used to be a huge thing. I
don't know that they still are. And Page Rage was this silly little
app that let you color up your Facebook profile which was, as they
are today, very uniform with everyone else's and all. And make it
look a lot more like a MySpace page, which was the big competitor
back then. People loved it.
Brad Miller (00:26:34):
That app exploded and we found ourselves in our second startup
together with a lot of big data from all those Page Rage
installs.
Kai Hankinson (00:26:43):
I think how did it go? Robert was like, "We just needed a better BI
solution." I don't even know if we knew the acronym BI at the time.
We just knew we needed a better tool set to help us understand this
data. We were using Excel for everything that was after Excel 97.
So that's when it went to like a million rows or wait, no. Am I
getting the years wrong?
Rob Collie (00:27:05):
2007.
Kai Hankinson (00:27:07):
Right.
Rob Collie (00:27:07):
Excel went to a million rows per sheet.
Kai Hankinson (00:27:10):
That was a game changer for Brad and me. I mean, we used Excel for
everything and it was limited to 70,000 rows or something prior to
that. But then we quickly outgrew the million rows and that's when
we started circling around and I ended up finding you Rob online.
And I called you and as it would turn out, you were the program
manager on our favorite software tool, that Excel 2007 version, I
think. Is that right?
Rob Collie (00:27:38):
Yeah. You see, this is great. You tell people the truth about
yourself, like what you did at Microsoft. And then you just sit
back and watch as your legend grows. There's an important word, he
says, "You were the program manager." Well, I was a program manager
on Excel. There were like 15 of us. Now, I was pretty senior on
that team and I had things I was in charge of. I was in charge of
the BI features in Excel, that kind of thing. But it does sound a
lot better when you say it. "You were the program manager for
Excel." And so when you introduced me that way and you've done that
now a million times with other people, I just let it go.
Rob Collie (00:28:18):
I mean, of course, why would I interrupt you while you're... I said
a program manager, you heard the program manager and you were
dually impressed. I was working as the CTO of my first startup at
the time. I had left Microsoft among other things, I had seen the
promise of this Power Pivot thing that we've been working on. And
so I was at a startup that was leveraging Power Pivot, as its
primary software. They told me as part of my employment deal, that
I was allowed to do some consulting. Okay, great. But I hadn't done
any, I hadn't actually done any.
Rob Collie (00:28:55):
And so you guys and you know this, but for the benefit of the
people listening, you guys were the absolute first clients of P3.
And I didn't tell you that at the time. In fact, I even tried to
get you to hire someone else instead of me, that's how good of a
businessman I am, right? Like, "Nah, you don't want to hire me.
There's a guy in San Diego who'll do a great job." And so I just
point you his way. I tried to have you not hire me and you hired
the other guy, didn't you? It didn't go well.
Brad Miller (00:29:26):
Hired and fired.
Rob Collie (00:29:28):
You called me back and said, "No, really, you should take our
money, Rob. Really, you should take it." And I said, "Okay, fine."
And I was just absolutely stunned at what you guys were doing. It
was just the three of us in a conference room for two days. There
were other people in the office, around and about, but it was just
the three of us the whole time. I remember this moment where it
started to dawn on me, what you guys were doing. And the scale of
it, when we were looking at the number of active add-ons. The
number of Page Rage add-ons that were out in the universe, calling
home, letting you know they were still alive.
Rob Collie (00:30:09):
I don't remember what the number was, but it was like double digit
millions. In 2010, it was October of 2010, something like 60
million. Was it 60 million? Was it 100 million? I don't remember,
but it was just I had never heard of it. I had never heard of Page
Rage. And yet there were like 100 million humans or 100 million
devices out there in the world running your Page Rage software. I
was just like, "What?"
Kai Hankinson (00:30:37):
Yeah. It was some mind boggling scale. It never ceased to me, how
big the world is. It's hard to wrap your brain around it.
Rob Collie (00:30:46):
Well, wrapping your head around how large the world is, is one
thing. Wrapping your head around, there's a reasonably good chance
that if I meet a human being on the street, they're running my
software, that's a different thing. You guys were paying what? You
paid Snoop Dogg to tweet about how good his Facebook page was
looking today, thanks to Page Rage and stuff like that, right? I
mean, you guys were doing something next level. It certainly got my
attention.
Kai Hankinson (00:31:15):
We had just got a written, the code tails basically, of Facebook.
Because, this was right when Facebook was just exploding. And so we
were writing their code tails. I think at our peak, we got up to
250 employees. And it was fast, it happened in a matter of two,
three years. The Snoop story is funny. That whole influencer model
is a lot more prevalent today than it was back then. But he was one
of the OGs back in the day. And you would pay him to tweet
something out, but he would make you come up with the tweet. So we
had 50 white software engineers in San Diego, coming up with
Snoop's tweet, which was pretty hilarious.
Rob Collie (00:31:59):
I'm sure it sounded very authentic.
Brad Miller (00:32:02):
We had experience passing that touring test, though.
Rob Collie (00:32:05):
Yeah. There you go, there you go. Are you Snoop? Check here if yes.
So the financial transaction in that engagement, went my direction.
Money came my way, you guys paid me. But I definitely benefited
more from it than you did.
Brad Miller (00:32:20):
That's an understatement, Rob.
Rob Collie (00:32:21):
Yeah, I know. We'll get to that. We're going to drag my name
through the mud, don't you worry, we're going to do a thorough job.
I have a saying that I have been saying now for 10 plus years.
Which is that, human beings do not know what they need until they
have seen what they asked for. And the very first time this thought
occurred to me was when I was working with you guys. We sat down
with what has ultimately become our jumpstart methodology. Which,
boil it down in layperson's terms is really just, just get started,
is the philosophy.
Rob Collie (00:32:56):
Which we loaded a bunch of your data and the Power Pivot engine,
the DAX engine, you did a great job chewing up tens of millions,
hundreds of millions of rows. It was great. It was doing everything
that you wanted in that regard. We got to a point where we had the
chart, the pivot table, whatever that you guys had wanted that
you'd described from the beginning. And then as soon as you saw it,
you're like, "Ah, no. That's my not quite right. We need to blah,
blah, blah, compensate for this, filter, that." That kind of thing.
And we tore through four, five, six, maybe even 10 iterations like
that in rapid fashion.
Rob Collie (00:33:33):
And got you to a place where you were looking at something that you
wouldn't have even known to have asked for. And it was clear that
you guys were really sharp, so it wasn't a failure of brain power.
It was just a basic limitation of human capability. You need to see
something that you asked for and then realize, this isn't good
enough. There was one other thing that happened at the end of that,
which was that you guys realized at that 10th iteration or
whatever, which didn't take very long. That, "Oh my gosh, we're not
even collecting the data that we need to answer the problem."
Rob Collie (00:34:10):
You probably don't remember this, but it was a really formative
moment for me. Where, you guys realized that you needed to have an
extra column of data that wasn't being reported by the add-ons. So
you had to go back and modify the add-on code. So that in the
future ad, when they called home and sort of reported on their
health or whatever, their activity. That you'd have the raw data to
do what you needed to do. And it was just crazy to me and how fast
that happened in the course of those two days. We went from
thinking you needed X to realizing you needed X to the 10th
derivative.
Rob Collie (00:34:45):
And then even that going, "You know what? Not good enough." These
ever rising standards, the ability to be greedy and truly chase
some thing to the end of where it needs to go and watching it play
out. And it just had never, ever, ever worked like that in
traditional BI. In traditional BI, you don't even make it to
derivative two, you just never get there. You get exhausted, you
get worn out, there's just too much time, too much expense. And you
certainly never get there with regular Excel either. Because, of
all the elbow grease that has to go into every single thing that
you do.
Rob Collie (00:35:21):
And I was just like, I left there even more in and I'd already
chosen to bet my career on this stuff, right? But I left San Diego
even more in. But of course, it didn't take with you guys, right? A
week later, how long was it before you stopped thinking about Power
Pivot?
Kai Hankinson (00:35:41):
We just couldn't do it without you in the room. I mean, it was just
so hard for our company to adopt those early stages. And I don't
know if it's because the product was a little bit immature, it's
evolved quite a bit or we had a shitty teach.
Rob Collie (00:36:00):
You mean the first guy before I came out there? I agree. You
totally did.
Kai Hankinson (00:36:04):
I don't know. I'm thinking about your second consulting engagement.
It didn't take the first time, we're like... But you sold us on the
revolution, you sold us on the vision. We wanted what you were
selling. We just didn't pick it up on that first trip. So we
brought you out for a second trip and it was a two day consulting
engagement. And the first day happened to coincide with the day I
was throwing a company party at my house.
Rob Collie (00:36:32):
Oh no. Hold on now, you've got this all wrong. You're talking about
the third time that I came out to try to make it stick with you
guys. There was already a second time-
Kai Hankinson (00:36:43):
Okay.
Rob Collie (00:36:43):
In between. You're talking about failure number three.
Kai Hankinson (00:36:47):
Okay.
Rob Collie (00:36:48):
The real problem with you guys and your adoption of this, was that
you just had too many resources. You had too little time, which is
common to everybody. You had too little time to learn something
particularly new, right? If you had to learn Excel from scratch on
the spot, it would be hard to take that as well. Takes years to get
good at Excel. But the other thing is that you just had developers
laying around, left and right. You just had too many technical
developers that you could go to them and say, "Hey, I want you to
write some code to take this data and turn it into a chart."
Rob Collie (00:37:21):
And it's like addiction in a way. It always seems like it's the
cheapest move, to go ask the developer that you've already got to
do something for you. And it is. At that moment, it is the short
term cheapest move. But you're stuck in this relative minimum.
You're just stuck in this gravity well.
Kai Hankinson (00:37:39):
Yeah.
Rob Collie (00:37:39):
Where, if you do take the time to escape it, you end up in a much
different place in a much better place. But I just couldn't break
you guys of that, we're surrounded by developers crack pipe. I
didn't have the right treatment plan for you. So that was the
second mission, right? I came out the second time and you had even
more people there. And my job was to convince, not just you, but
also the new guy. And so we did that, he wasn't really all that
convinced. And then you still had the developers. Because, you guys
just kept getting back on that horse.
Rob Collie (00:38:11):
You kept falling off and getting bruised and then you're like, "Oh
we should call them out again." And I'm like, "Okay, great. Let's
go. I always enjoy visiting you, let's do it." So yeah, the
longtime blog readers have even seen a picture of the shame that
Kai's about to describe. If you've been around reading the Power
Pivot Pro blog from the beginning, I sneaked a picture in of this.
So go ahead Kai, embarrass me.
Kai Hankinson (00:38:38):
I'm sure you had a lot of anxiety. This was your third attempt, you
had failed miserably the first two times with your first client. We
invited you out to my house with the whole company and I'm just
going to leave it at, you drank more than anyone in my company.
Rob Collie (00:38:57):
Lifetime.
Kai Hankinson (00:39:01):
You weren't anxious at the end. But I ended up having to drive you
home and then pick you up the next morning to come back for day two
of your consulting engagement.
Rob Collie (00:39:10):
In my rental car, by the way, right? You picked me up the next
morning in my rental car.
Kai Hankinson (00:39:16):
When I saw you that next morning, I understood that day two was
off. You were just going to be physically present.
Rob Collie (00:39:26):
And you paid me anyway.
Kai Hankinson (00:39:28):
It was the right thing to do.
Rob Collie (00:39:31):
And you invited me both days actually that I was there, to your
after or noon KPI meetings. Which again, I learned something you
were paying me to learn. It was awesome, I really appreciate that.
You guys are the best. You had like 20 charts that you looked at
every single day at four o'clock in the afternoon. And it was the
same 20 charts, every day and there were like 10 people in the
room.
Kai Hankinson (00:39:57):
About 20, easily.
Rob Collie (00:39:58):
Okay, yeah. It was a big room.
Kai Hankinson (00:40:00):
Huge meeting.
Rob Collie (00:40:01):
And there was no interactivity, you couldn't drill down on a chart.
That's why there were 20 charts. It's because you probably had
started with five charts.
Brad Miller (00:40:08):
There were probably 50 times four. There were probably 50 slides
with four on slides, probably 200 charts.
Rob Collie (00:40:17):
Right. You didn't start with 200 charts on the first day of the
KPI. There's 10 charts and you're like, "Ah man, these don't answer
the questions that we need to follow up." So clearly the answer is
more charts. And this is how, by the way, traditional reports
explode in traditional BI as well. Like, you got a question? You
need a new report. Got a question? You need a new report.
Organizations sometimes end up with tens of thousands of reports.
Many of which have only been looked at once. Some of them have
never been looked at, they were asked for built and never used.
Brad Miller (00:40:48):
Yeah.
Rob Collie (00:40:49):
And it's all super slow,.but you guys, I was really, really, really
rapidly against this. This idea of you'd have 200 static charts. It
was in some sense, offensive to me. Because, I was so big on and I
still am... On the concept of interactivity in the ability to
dynamically ask questions, as opposed to having to build all the
charts to answer all the questions, which is a losing game. You
never get there. At the same time though, it was a good meeting. I
could not say to you that it was a bad meeting or that it would've
been better. Because, it was so weird to me that it was the same I
guess 200 charts on Thursday that it had been on Wednesday.
Rob Collie (00:41:36):
How much could change in your business in one day that you would
look at the same charts? That the conversation was completely
different both days.
Kai Hankinson (00:41:44):
Yeah.
Rob Collie (00:41:44):
It was super valuable.
Kai Hankinson (00:41:46):
Rich conversation every-
Rob Collie (00:41:48):
Yeah.
Kai Hankinson (00:41:48):
Every day at that meeting. I mean, it would just be like either the
charts would be stimulating. The changes in the charts would be
stimulating the conversation or we'd come to the meeting with
business issues that we would then turn to the charts to help us
think through.
Rob Collie (00:42:04):
Yeah.
Kai Hankinson (00:42:05):
And a quarter of that room was dedicated to supplying that product.
That's how it went from five slides to 50 slides. Is like almost
every meeting we would put in a request that we'd usually be
looking at the next day.
Rob Collie (00:42:20):
Yeah.
Kai Hankinson (00:42:21):
Because, we had so much expensive manpower thrown at the
problem.
Rob Collie (00:42:25):
I mean, you were in essence, an IT organization from top to bottom.
And so when you have that many resources laying around at your
disposal, you tend to take that path. But I did, I learned
something from that as well. I came back from that experience,
having seen the... I also learned, don't eat chocolate cake,
birthday cake at the office and that'd be the only thing you eat
all day and then go straight to the Cinco de Mayo party at the
CEO's house. That's another lesson, don't get into land war in
Asia, et cetera, et cetera. But I came back, I realized that I had
been too much into the concept of interactivity.
Rob Collie (00:43:06):
I refined something about my stance about all of this. Which was
that, any report you produce needs to be useful before you touch
it. Before you interact with it, it needs to already be starting
the conversation. Whereas before, I was being lazy essentially and
I was saying, "Look, if the thing is interactive and it can answer
any question, it can just sit there like a slug at the beginning of
your relationship with it every day. Staring at you saying, 'Yeah,
I know I'm telling you something that doesn't matter to you, but
you can ask me the things you really care about.'"
Rob Collie (00:43:40):
And so it was almost like interactivity was required before you
could derive anything from a lot of the stuff that I had been
producing. That's much closer to where I am today, by default
useful. I'm not as obsessed with interactivity, it's table stakes
that you'd be able to drill down. But if you have to interact with
it before it tells you anything at all, it's a bad design. And so
again, you paid me, I drank your booze, I came home wiser and you
guys didn't adopt it.
Kai Hankinson (00:44:10):
Yeah. I mean, you certainly helped convince us that we were on the
right path to be using the big data, to answer our questions. And
we were sold on the vision that you were high priesting.
Rob Collie (00:44:21):
You say 'we', but let's be clear. You weren't both as equally in,
were you? One of you is a little bit more skeptical than the other,
that's true, yes?
Brad Miller (00:44:31):
I think that's true, but I will give you some credit though. I will
say that, you have two main parts to your business, right? And we
talk about this a lot when we work on the search advertising for
you guys. You have your training side of the business and you have
your consulting side of the business. And so, when we kept bringing
you back, it wasn't just for your fun personality. It was for the
consulting side of the business. And so while we weren't yet sold
on being trained on the tool, there's a whole side of your guys'
business, where you do provide a lot of value just through
consulting.
Brad Miller (00:45:11):
You guys are experts in your field and so when Kai and I are able
to sit in a room with you and talk to you about data and BI for
hours at a time, that is super helpful to us, tool or no tool.
Rob Collie (00:45:25):
I appreciate that, that truly actually really is flattering. I
always felt like I was learning, interacting with you guys more
than you were. And that's just sort of the nature of imposter
syndrome, but you're right. That's another thing that's different;
if you all met our company today, things would go differently. Even
if you were in exactly the same spot that you were in, in 2010.
Things would go differently today because at the time, I had a day
job, right? I couldn't devote a whole lot of time to you. So it was
just like, "Look, I'll come out and tell you some things and leave
you some stuff.
Rob Collie (00:45:55):
And then you're on your own." That's how the company started. The
first three years, I was still full time employed elsewhere. But
now this is what we do. This is what we do all the time. And so we
would have the time to dedicate a consultant or maybe even multiple
consultants to helping you on an ongoing basis. Even with fractions
of their time, it wills till be a lot more than what I could have
allocated back then. And we would just do stuff for you now.
Brad Miller (00:46:21):
Yeah.
Rob Collie (00:46:21):
There'd be more continuity there and I think it would've gone a lot
differently.
Brad Miller (00:46:25):
Yeah.
Kai Hankinson (00:46:25):
Was it next that you got accidentally added to Brad's bachelor
party invitation?
Rob Collie (00:46:30):
Yeah. He's been really kind about never admitting that mistake.
Kai Hankinson (00:46:33):
And I'm pretty sure it was on purpose. But you were convinced it
was an accident that you got invited, to Burning Man.
Rob Collie (00:46:39):
Honestly, the reason why I was like a Tinder box ready to explode
on that fateful Cinco de Mayo, is just, it wasn't just a
professional decision leaving Microsoft. I'd been through some
really, really tough divorce and family stuff. And I was away from
all of my friends and I wasn't in Seattle anymore, I was in
Cleveland. Essentially by myself and hanging out with you guys
every time you guys would bring me out there. Hanging out with you
guys was the best thing that had happened to me in years, it was
just great. I've told you guys this, but I would have dreams in
which I was out there working with you guys, living out there,
hanging out with you. That was how much, that's how strong the pull
was.
Rob Collie (00:47:24):
Those were tough years. Out there hanging out in San Diego of all
places. Leave Cleveland, get off the plane in San Diego and be
like, "Oh my God." And then it just felt like living a completely
different life. "Oh yeah, of course. I'm going to go to the CEO's
house for Cinco de Mayo party. It's on the Pacific ocean?" "Yeah.
It's on the Pacific ocean. We're just hanging out in the front yard
on the Pacific ocean." "Hand me that margarita." Everything was
predestined from that point forward. I mean, I was in need of an
outlet and boy, did I find one. When I got the bachelor party
invite, I mean, it was just too good to be true in a way,
right?
Rob Collie (00:48:07):
Brad wants me to go a Burning Man with him? And there were so many
people on this email. I remember it being, I just assumed that Brad
has 1,000 friends, it probably wasn't that many. But I just
remember it as being like 300 people on the email or something. And
so it was so easy to fat finger that and include the wrong Rob or
something.
Brad Miller (00:48:32):
I definitely did not include the wrong Rob. There were a lot of
people on the email because I wasn't sure what the response was
going to be when I had my bachelor party at Burning Man, because it
was 2015. And so, I just invited everyone from my life, going back
to high school that I really liked and really wanted to be there.
Even if I hadn't seen the person in 10 years, I wanted to invite
them. Because if they wanted to go, I wanted them to be there. So
there were a lot on there, but they weren't 300.
Rob Collie (00:49:04):
Okay. And this is really something I appreciate about you, Brad; is
just, you're really good at setting aside everything that doesn't
matter. And just focusing on what would be the most human and fun
thing? You don't get hung up on, "Oh, is it weird to invite these
people, I haven't seen you in a while?" You just set that aside.
That didn't get in your way.
Brad Miller (00:49:28):
Yeah.
Rob Collie (00:49:28):
And so then I did the responsible thing and not read your emails.
As soon as it started to become real, I started to get terrified.
Like, "Oh my God, I'm not Burning Man material." And you talked me
into it.
Kai Hankinson (00:49:41):
What a trip.
Rob Collie (00:49:43):
Yeah.
Brad Miller (00:49:44):
[inaudible 00:49:44] with 18 of us going and only one had ever been
before. He had only been one time with his 80 year old dad back in
the day.
Rob Collie (00:49:53):
That was when we finally dispensed with the idea of, this isn't a
professional client relationship. This is a friendship.
Kai Hankinson (00:50:03):
That saved us...
Rob Collie (00:50:03):
Well, client relationship this is a friendship
Brad Miller (00:50:03):
That saved us a lot of money.
Rob Collie (00:50:05):
It did. A lot of failure, you got out of the Facebook business.
Facebook had something to do with that, right? You were riding
their coattails. One day they turned around, and said, "Hey, get
off my coattails."
Brad Miller (00:50:19):
Yeah. They did not like our business model. They did not like us
covering their ads with our ads, that was coming to an end, but
before we ended up selling that company, and we had a nice exit
from that one. We had these 60, 100 million add-ons that had
downloaded our software. We were looking for ways to pivot, we were
looking for ways to expand, and so at some point, Kai said, "Brad,
I'm giving you three or four resources. I'm sending you down to the
basement for three months. You're going to do essentially a
consulting project, and figure out where we're going next. You're
not allowed to talk to anyone else in the company, you just of
focus on this." We went down, we spent three or four months on it,
and we came up with two ideas.
Brad Miller (00:51:08):
One was a data analytics company, because we just had so much data
coming down from these add-ons that like, there was something to do
with this data. We ended up exiting that company, and selling it to
a company in New York. Then the other spinoff was what's today
Agree Media, which is in the lead generation space, having so much
data, one of the things we could do with that data is identify
possible leads for various industries and verticals. That was the
original seed of Agree. After we sold off the main company, the big
company, Kai, and I bought out our other partners, and so now we're
just owners in Agree, 50-50.
Rob Collie (00:51:50):
Yeah. That timeframe, when you were in the basement, I remember
this because this is when I was out there, when I was
contemplating, and in the final stages of contemplating leaving the
startup I was at and turning P3 into its own distinct entity full
time commit to it, that kind of thing. This was in like 2013. Does
that sound about right? I just remember when I was out there
pitching you guys on being an investor in my company, which in the
end we decided we didn't need any investment, but you guys had such
great success pitching your friends and family on investing in your
companies. I just said, Hey, I'll return the favor. Agree Media.
This is why we're here. What is it? We've taken an hour to get
around to the crux of it.
Rob Collie (00:52:39):
When I talk about you guys in Agree Media, to the people who don't
know you, I describe you as experts in data driven advertising.
People who know the internet advertising space really better than
anyone is how I queued it up. You say, it's lead generation. You
didn't say, Hey, here's the idea. Let's get really good at
advertising. Let's get really good at digital advertising. Let's
get really good at digital marketing. That wasn't the goal. That's
not what you came out of the basement with. Right. You came out of
the basement with, we should be a lead gen business.
Brad Miller (00:53:16):
Yeah.
Rob Collie (00:53:17):
Tell us a little bit about the advertising operation.
Brad Miller (00:53:21):
Yeah. I mean the lead generation business is interesting because
one example where it struck home with me was like, you go to
various conferences for whatever industry you're in. Right? With
the prior software development company that we were working at,
that we owned, we'd go to these conferences, and it would be guys
in their early twenties, and hoodies, and the whole Mark Zuckerberg
look right. Everyone at the conference, the first lead generation
conference we went to, the big one is called LeadsCon. We show up,
and everyone's either in a suit or a sports coat with tie and
shirts. We're like, what's going on here? They're like, we've never
seen this before. I've never been to a conference, like where
people aren't in hoodies.
Brad Miller (00:54:10):
It turns out that the lead generation business had been around long
before the internet, right? I mean, guys were sending out flyers in
the mail, and there was a bunch of other ways to advertise
pre-internet, and lead generation business has been around forever.
That's the Genesis of where most of those folks had come from.
Whereas our background is much more finance, quantitative analysis,
software development, artificial intelligence that kind of thing.
Instead of coming at it from a marketing bent, we're coming at it
from that data analysis AI bent, which has given us, I think, a
competitive advantage in the industry. We use that primarily on our
own in-house proprietary systems, as well as when we're hooking up
with Google, Facebook, Instagram platforms.
Rob Collie (00:55:03):
I've never heard that part before. That's new to me. It reminded me
of, have you seen Glengarry, Glen Ross?
Brad Miller (00:55:10):
Not in a while, but yeah.
Rob Collie (00:55:11):
They now disgraced Kevin Spacey, is holding like a box in his hand
and says, "No, these aren't, those other leads, those crappy leads.
These are the Glengarry leads. You're not worth it. I'm not going
to even share them with you. Because you're so rookie." What a
hardcore movie that was really. Yeah. I'm pretty sure that the
Glengarry leads did not come from a digital system. I'm pretty
sure, they came from some other shady thing. I have no idea what it
would be, but he holds them like they're gold.
Brad Miller (00:55:44):
That's the other thing we love about the lead gen business is that
other forms of advertising, whether it's television, billboards,
print, whatever it was, it was like, there was no real way to
quantify the success of those campaigns. Right? Most of where the
money was made was like, you had a good salesman who was taking
people out for lunch, and drinking martinis mad men style, and
convincing them that their campaigns are working. They were rough
numbers. You really couldn't see anything down to any level of
detail, obviously, because it's all offline. In the lead gen
business, it doesn't require much sales, sorry, Tim, our sales guy,
I don't mean to throw you under the bus, but like Kai and I, that's
not our forte. Right. In the lead gen business, if the numbers back
out, and you're sending the quality, you will get the bigger order,
and they will pay more. If the numbers don't back out, they won't.
Taking out that salesy middle man was really a feeling to us.
Rob Collie (00:56:50):
This is a theme across all business, right. Is that in the world of
offline lead gen where you just have a good set else, person, you
can't lose. If you're the lead provider, if you're the lead
consumer, you can absolutely lose. You'll never know it, but it's
like, I got you some leads, and your business improved. Well that's
me. I did that. Oh, I got you some leads, and your business didn't
improve. Oh man. Imagine how much worse it would've been. If I
hadn't gotten you, those leads he just got a story for everything.
A lot of people are even though no one wants to talk about it or
admit it, there's still a lot of very important decisions being
made in the business world every day that are basically done like
that. There's still a lot of gold rush opportunity everywhere.
Brad Miller (00:57:35):
Yeah, probably about, I don't know, two or three years ago for the
first few years of the company, we were primarily Facebook in terms
of our advertising. We were strictly social advertising, Facebook,
and then Instagram, maybe two or three years ago, we realized that
we had to diversify into Google search for two reasons. One for
diversification, two for quality. Facebook is much higher in the
funnel. You're hitting someone up in their newsfeed and they're not
looking to go back to school at that moment, but something catches
their eye, and then they might go check it out. Whereas, Google
someone is searching. I want to go back and get my cosmetology
license. There's a ton more intent there, much lower in the funnel.
The quality of those leads are much higher. We're primarily in the
for-profit EDU space. Education vertical, so our customers are
names, University of Phoenix, DeVry Art Institute, those kind of
schools. They were looking higher quality, and so we transitioned
to Google. Now we're about, I'd say 20% Facebook, Instagram, and
about 80% Google. We used to be Google AdWords. It's now Google
Ads.
Rob Collie (00:58:52):
You guys have put Sally Struthers out of a job. Remember those old
ads on TV where she would go, "Do you want to make money? Sure. We
all do." Then she'd run through all of these things that, there was
always gun repair, gun repair was always one of the options that
was on that she was advertising. You talk about a primitive model,
right? How did they ever, that these ads worked, but boy, we saw
them for years. Didn't we?
Brad Miller (00:59:17):
Yeah. We gave you what, like your fifth crack at us, at Agree. We
like, come on, Rob, come sell us again on this vision for Power
BI.
Rob Collie (00:59:26):
That's right. I mean, actually, even at Agree, there were multiple
failed attempts. We'd given up on the notion of it being a
financial transaction, right. It wasn't how it worked. I'd just
come out there, and hang out and we'd spend some time and chit
chat. We'd both learn that kind of thing. I mean, even once we get
to Agree, there's at least two or three totally aborted attempts at
trying to get this stuff to take. Then Brad goes off and he finds
self this thing called Periscope it's just traditional reporting
with a slicker new look on it. You guys brought me out. I remember
saying, "Hey, can we replace these Periscope reports with Power
BI?" I'm looking at them in like on the average dashboard, like
three of the modules on the dashboard we're reporting, oh, sorry.
We failed to load the data in time we timed out.
Rob Collie (01:00:15):
I'm Like, I could probably replace these, but I'm might have a hard
time with replicating those performance failures. My stuff's just
going to work. All the stuff's just going to return the answers. I
suppose we could roll some dice or something behind the scenes, but
because you guys now are, are using Power BI very aggressively.
Brad Miller (01:00:32):
Every day.
Rob Collie (01:00:33):
Even here I don't get to claim credit for it. It was almost, we had
to maintain the joke or something. Like if you guys were going to
start using it, we would need someone else to have helped you so
that we can say that I still was O for 10 with you guys. Ultimately
I think Austin was really crucial and really nudging plus the fact
that the matter is, is that just because I was first. I was, I was
like the first professional in the world dedicated to what is now
called Power BI.
Rob Collie (01:01:04):
I was the only person that was like saying, this is the only thing
I do. My only source of money. I was first. That doesn't mean that
would never translate to being best. I was pretty good. I was
pretty good, but people we hire are better. I am absolutely the
worst at these tools of the people at my company. I think that's
great, takes a lot of pressure off me. Austin was definitely one of
those. He's better at it. He was able to make progress with you
quickly. His goal was to build stuff with you, which I never had
time for.
Brad Miller (01:01:37):
Yeah. Austin was key from your side, and then Sean, our head of BI
slash CFO was key from our side, so putting those two together,
really helped us turn the corner.
Rob Collie (01:01:52):
By the way, I love that head of BI slash CFO. That's the future,
business intelligence owned on the business side or at least driven
from that side, supported from the IT side. For sure. You guys
don't have a huge footprint of people, but that's your model you're
already running on the model that the world is inexorably heading
for. You're seeing so many more of these hybrid titles that you
just would never have seen before.
Rob Collie (01:02:23):
I think that's really exciting, and yeah, Sean has really, really,
really, really taken to this stuff. What are the sorts of things
that you look at in Power BI? Because you know, Hey, I was going to
ask the juicy softball question. I mean, but seriously doesn't
Google just provide you with all of the reporting that you need. I
mean, AdWords has got a reporting portal. Why do you need Power
BI?
Kai Hankinson (01:02:45):
What don't we look at in Power BI these days? It's really got to
the point where we almost exclusively, answer our questions with
Power BI. What we found is that, sometimes you'll have these
situations where a DBA could bang out a slightly quicker answer
with a sequel query, but we've learned the hard way that, if you
just put the little bit of extra time, and it's not much, but just
a little bit of extra effort it into answer that question with a
Power BI model, it will just pay dividends.
Kai Hankinson (01:03:20):
You will be able to keep asking it questions, and you'll be able to
ask the next version of the question, and they get easier and
easier to build each time. We are using Power BI for all our BI at
this point, and we love it. It's like we're finally living the
revolution you promised us back in 2010, it took us a long time to
get there.
Kai Hankinson (01:03:46):
One of the reasons the AdWords portal isn't sufficient is that, we
need to stitch that data together with what happens on our side,
what happens those ad clicks once they get to our website, which
ones drop off, where do they drop off? Which ones become leads,
where do they become leads in? Which schools are they matched to?
Even then we get the offline data that we have to collect and bring
back in and import into Power BI, which is, well, what happened
after we sold the lead, which leads applied at the school, which
leads enrolled in school, which leads actually started.
Kai Hankinson (01:04:21):
We get data all the way down to, they pass their first class even.
So we need to stitch that full conversion funnel if you will,
together. And AdWords only has some of it.
Rob Collie (01:04:32):
That is the thing so often is that it's so rare that there's a
report that's useful that truly helps you. That is sourced from
only one system. Anything valuable is almost always going to span
at least two systems. You're talking about just in that quick
survey, there were at least three systems and one of the systems
doesn't even belong to you.
Rob Collie (01:04:59):
It's your customer's systems that are reporting back to you about
like, well, you sold us a hundred Glengarry leads, but they
actually felt more like Glen Ross or vice versa, right? So you get
a quality feedback on a longer time scale. But, but you need to use
that to steer because if you deliver poor quality over time, your
customers are going to stop buying from you.
Rob Collie (01:05:22):
And so if you can't span silos with your BI, if you can't span
silos with your reporting, you're all always looking at one part of
an elephant going like, "Is this an elephant? I don't know. It's
gray. Could be a rhino, no idea.@.
Rob Collie (01:05:39):
And so I'm really happy that you guys have gotten to that. And you
and you operated at a pretty impressive scale. You guys can't even
use certain commonly available integration tools like stitch for
instance, because they have some cap that I'm sure when they set
that cap, the engineering team said to themselves, no one's ever
going to need more than this. They said it to be 10 times what we
think anyone would ever possibly humanly need, right? It's like
10,000 ad groups or something like that. Is there limitation and
you guys are like, please 10,000 ad groups.
Kai Hankinson (01:06:19):
We have some proprietary technologies that we developed to leverage
the AdWords platform. It's driven us to have to come up with some
elaborate structures over at AdWords that involve hundreds of
thousands of ad groups and campaigns keywords. And in order for
them to be able to support the kind of integration with our
technologies that are... we live by a philosophy of AdWords whereby
we think that the smart bidding technology artificial intelligence
is that Google has built is some of the smartest AI in the
world.
Kai Hankinson (01:06:57):
I mean, it is just so, so powerful at doing what we need it to do,
which is to go out and bid the right amount on the ads that we run.
Bid the right amount for the right user at the right time on the
right device, all those things. And we believe wholeheartedly that
if you're going to succeed, you have to be leveraging their smarts
to the max.
Kai Hankinson (01:07:20):
But then there's also a component of, there are things that we
know, not many, but there are things we know that Google doesn't
know about our business. We know that that customer just canceled
this other customer, close their calls there for labor day. We know
we just got a double size order on our Chicago school.
Kai Hankinson (01:07:40):
We've set up the mechanisms whereby Google's Smart Bidding will
learn those things over time, but that's just it, it's going to
take time. So we will elaborate technologies that us to leverage
their smarts while still injecting ours. And that's a tricky thing
to do because it's easy to clobbered their smarts and be like,
we're switching Google onto CPC bidding instead of Smart [TiBO
01:08:03] as bidding or whatnot.
Kai Hankinson (01:08:05):
But now you've just clobbered their smarts and you're not
leveraging the best of both. So, that's been one of our driving
philosophies there. That's driven the elaborate structures that
have taken us beyond some of the caps in many of the products out
there, like stitch being one of them, Stitch data.
Rob Collie (01:08:20):
I even love the names you guys have come up with for some of your
proprietary stuff. It is just as a terminology guy. I got mad
respect for things like probabilistic pixel and denominator, even
Trillions with capital T. I mean, it's hot. You guys have got good
naming game
Brad Miller (01:08:41):
[Redon 01:08:41], don't forget about Redon.
Rob Collie (01:08:43):
Ooh, I never heard about Redon. You we'll have to talk about that
later. I need to know is PG 13, is it Patrick Swayze? Is it, I
mean, just let's leave that. But, one of the things that's
impresses me about you guys and there's many things obviously, but
you guys are just so into the weeds, into the Twilight zone of all
of this. Your intuitions and everything. You've spent so many years
building them up and learning the way in building all this
technology and proprietary techniques and everything for optimizing
your ads, you still keep hiring experimentally.
Rob Collie (01:09:19):
You keep hiring third party ad agencies, ad optimizing services to
see if they can do better than you. I mean, so many times when I
visited you, like you've been on the phone and interviewing like
just some... Oh this is this other firm we're thinking about trying
out, just the humility and also just the stamina that it takes to
keep doing that.
Rob Collie (01:09:46):
And it's worth noting that you're still doing that. You're still
trying it. If you could get out of the business of managing ads and
just be like the leads clearing house, you probably prefer that in
a way, no one's ever come along that's that's been able to do
better than you. And so you just keep getting better at your own
stuff.
Brad Miller (01:10:05):
Yeah. I don't know if we would prefer to get out of it. I mean
people tend to do what they love. If I was kind of retired and
really rich and I wanted to spend a few hours a day doing something
like this is probably what I choose to do, right. Is to dive into
some huge, powerful platform and start pivoting stuff and figuring
out. It's my version of a, that's the way I would spend my hobby
time. It's like, but I get to do it at work.
Kai Hankinson (01:10:37):
I love my job. I mean, one of the greatest things about working at
Agree is we're only a team of 12 right now, but we've pulled
everyone forward from a prior company. In some cases we pulled them
forward from two prior companies. So it's the best of the best that
we get to work with.
Kai Hankinson (01:10:56):
We've probably employed like 600 people together over the years, or
maybe more. And these are some of the 12 best. We get to spend all
day every day, solving fun problems with good people. It's a dream
come true. I've learned over the years with our startups that we
both love and do best at a specific type of business challenge. We
do best when it's like a high volume of transactions with a quick
feedback loop because those are the pro that you can just really
work the data to kill.
Kai Hankinson (01:11:31):
It's like, I need that quick feedback loop or else... is just going
to be such a slow cycle of improvement. And I need the high volume
of transactions to drown out the luck factor. I learned that at the
poker company, I just couldn't to believe I used to love table
poker prior to running that poker company. And it almost ruined
table poker for me, because we would see these runs, these bad luck
runs that would just blow your mind. We would have different
versions of our software. So we'd be like running version 53.2 for
a week.
Kai Hankinson (01:12:07):
And kicking, come Monday, all of a sudden what happens? We're
losing money. Why are we losing money? We're just spending all day
diving to the data. This doesn't make sense. How can we be losing
money?
Kai Hankinson (01:12:18):
We're running the same version that crushed last week, nothing's
changed. We're checking all these things. Tuesday get killed again.
This doesn't make sense. We made money every day. Wednesday, all of
a sudden, after 300,000 hands, our luck turns. It goes back to the
whole way of just churning out the profit day after day. Luck is
such a factor and you need that high volume of transactions to
drown it out. If my success depends on whether or not this one big
commercial sale closes this quarter, Ugh. I just don't like
sleeping with all that.
Brad Miller (01:12:54):
To answer your question from earlier, we're currently bidding on 17
million keywords in 2 million ad groups.
Rob Collie (01:13:01):
Wow. Yeah. 10,000 is cute. Isn't it? It's so cute. Yeah. If you let
us have 200 accounts, merge them somehow we can use your service.
So let's see if we can check the check boxes. First of all, you
guys are like, "Ah, I don't want to be in any sort of hourly
business," like that professional services business. So that's
something you want to avoid. You want tens of thousands of trials a
day and you also need almost a near immediate feedback. You have
now described the exact antithesis of our business at P3.
Brad Miller (01:13:44):
I got one more for you that'll fit in that category too.
Rob Collie (01:13:47):
All right. All right. Bring it.
Brad Miller (01:13:49):
When we developed the poker software, we could have licensed it to
someone else. Right. And let them play poker with our software, but
we decided if it's that good and we're building it, we might as
well use it to trade ourselves.
Brad Miller (01:14:04):
And so up until now, and I know we're working a little bit together
with you, but up until now, we're like, "Look, if we're that good
at AdWords and Facebook and digital marketing, let's just use it
ourselves as opposed to doing it for someone else."
Rob Collie (01:14:16):
That's right. Yeah. We've been sort of inching our way towards as I
say it finding a reason to risk our friendship. That's what you do.
When you get into business together. You want to ruin a personal
relationship. You really want to ruin it like permanently. I
suggest going into business together. And so like moths to flame,
we've been slowly inching towards this really slowly, but at least
with our eyes open.
Rob Collie (01:14:42):
So we've been collaborating, we've been providing Agree with
additional Power BI support and you all have been helping us by
architecting and advising, et cetera, et cetera on our advertising
campaigns, which is somewhat unusual for professional services
firm.
Kai Hankinson (01:15:00):
Who did you send to help Agree? Of course...
Kai Hankinson (01:15:03):
This is firm.
Rob Collie (01:15:03):
Who did you send to help agree? Of course, your woofer.
Kai Hankinson (01:15:05):
That's right. We sent Paul. We're like, "Okay, we've got an in
here. These are two guys that know how to dumpster dive in
Hawaii."
Rob Collie (01:15:15):
[inaudible 01:15:15] has been great. He's really taken us to the
next level.
Kai Hankinson (01:15:19):
Well, like you said, you've hired 600 people over the years and
you're down to like 12, that's essentially like our interview pass
rate as well. We're like in the 1%-2% range.
Rob Collie (01:15:31):
Wow. And so, that's that similarly filtered group. We all say it,
we're the only office we would ever go back to. And we're all over
the country. With a 1% pass rate, you can't hire in one spot. You
can't say, "Oh, it's got to be in Indy, wherever I happen to be
living at the moment." People all over the country, full-time
employees, no office. And if there were an office with all these
people in it we'd actually want to go to it. But yeah, we've been
saying, "Hey, let's take your advertising expertise and let's apply
it, both to help the P3 business but also as a completely different
test case than the industry that you've been in."
Rob Collie (01:16:13):
Let try it with when there isn't the 17 million keyword volume.
There isn't that massive volume. Let's try it. When there isn't
instant feedback. We're not running an e-commerce business. It's
ultimately a relationship business. We have this thing at our
company. We like talk about how people say nobody believed in us.
Nobody gave us a chance. I don't know how many times I've been told
you can't run a professional services business this way. You can't
start relationships over the internet, but we can because once you
try us you understand that there's nothing like us.
Rob Collie (01:16:50):
How do we get people to try us? How do we get more people to try
us? And it's been very effective for us, for our business model.
It's not been easy. And the whole time we were talking about all
these hard lessons you had to learn about bad luck, and needing
volume, and all of that. They're really echoing in my head because,
I think Brad's favorite way to start a sentence with me is
something to the effect of, "Well, you'd think that would be a good
idea, but..." Right? This is what I do, I come up with ideas or
suggestions and then, way more often than a coin flip would
indicate, my ideas are bad.
Rob Collie (01:17:28):
And so, because that subject matter expertise that you... You're
not just looking at spreadsheets, or Power BI reports, and turning
knobs like robots. You're having to blend this with, what's
basically, a decade and a half of experience being in this digital
marketing world. This digital world. It's hard. It is hard. And I
know that I take a toll on Brad with my, "Oh, come on." Right?
Like, " Shouldn't we do X, Y, Z?" And he's just so exasperated with
me.
Brad Miller (01:18:05):
Yeah. What we spend most of our day doing, or at least what I spend
most of my day doing, is trying to figure out what's going on in
the black box over at Google. Those guys have built something
that's essentially taking over the world. The robots are coming. No
one really knows. No one knows what they've built over there. It's
taken hundreds of engineers and a bunch of marketing people to
write up the documents. But we've done several trips over there and
tried to track down someone that has a thorough understanding of
what's going on under that hood. And the AI is just, it's taking
over. So it's kind of figuring out what it's doing and trying to
play nice with it.
Rob Collie (01:18:53):
I read recently there was like some AI that was able to look at a
scan of like a retina or something. Was this something you guys
told me about? You look at your retina or something and tell you...
It was something ridiculous, it could tell you whether or not you
had cancer or something.
Kai Hankinson (01:19:10):
Gender?
Rob Collie (01:19:11):
Maybe it was gender. But the point is the human beings looking at
it, even though it was deadly accurate, no one could understand
what the AI was seeing.
Kai Hankinson (01:19:19):
Yeah.
Rob Collie (01:19:20):
The human beings that built it don't know how it's doing it. And
this is where Skynet comes from. Right? One day Google becomes
self-aware, the AdWords platform, and it starts turning the entire
earth into a giant conversion funnel. Then we're all paperclips,
all the way down.
Brad Miller (01:19:43):
It's interesting to note the differences between Google and
Facebook. And I don't know if we have time or interest.
Rob Collie (01:19:49):
Sure. Why not? You do operate on both platforms and they have
similarities, but they're of course, very different. Probably more
different than similar. What does that end up looking like for
you?
Brad Miller (01:20:01):
It looks like a lot more work on Google. We'd probably spend 95% of
our time on Google and 5% of our time on Facebook. Or even less.
And that's just based on the design of the two products. They're
both so sophisticated and their AI is so advanced, but yet they've
implemented in a different way that I would say probably 10 years
ago, 5 to 10 years ago, marketing was a very different beast. It
was all about, "Okay. I got to figure out who my demographic is. I
want to target males 25 to 33, with incomes above X, that live in
these zip codes." And whoever could identify their demographics
correctly and bid accordingly that was the name of the game that
was who won. Now, especially on Facebook, there's none of that
anymore. Some of the old school agencies are still doing it that
way, just because that's all they know and that's what they've
grown up with, but it really doesn't make any sense.
Brad Miller (01:21:02):
So like at Facebook, what we do now is we advertise, someone clicks
on our ads, they come through our flow, we get a conversion, we
then send back as much information about that conversion as we can
through Facebook. Facebook then goes and figures out who to target
using their algorithms and their AI. They then figure out who is
most similar to that person that just converted for me. So I have
no idea, even, who they're going after and no one at Facebook does
either, but based on all the hundreds, thousands, millions of
different data points that they analyze, they're able to better
figure out who to serve that ad to than I would ever be able to
just going after males 25 to 32, when incomes over X. So a lot of
what we do at Facebook is just content creation, coming up with new
ads, different ad copy, that kind of thing. And then you just kind
of throw it in and let Facebook work magic.
Kai Hankinson (01:22:08):
Can I jump in, there's something about this Facebook thing...
Another thing I've learned from you guys that sort of stuck with
me, is this notion that if you're going to try something like a new
campaign or something like that, and you create 10 different ads
that are just 10 different candidates to try to achieve the same
thing. You just try 10 different content strategies. And then you
play this game with yourself of predicting ahead of time, rank the
10, how well are they going to perform?
Brad Miller (01:22:33):
And your ability to predict which ones are going to perform or is
almost worse than random.
Kai Hankinson (01:22:38):
Oh yeah.
Brad Miller (01:22:38):
The end result is closer to the inverse of your guess than it is to
your original guess.
Kai Hankinson (01:22:45):
Yeah.
Brad Miller (01:22:45):
And just really shows how little we actually know and how little
you can know. And the value of experimentation. And, as you said
before, the value of letting these massive overlord AIs do what
they're good at, which no one understands.
Rob Collie (01:23:05):
Yeah. The whole notion of AB testing, for example. If Facebook
becomes irrelevant because, well, a lot of times where people get
tripped up is, let's just say, I'm testing two ads... Well, let me
give you a different example. So one of the programs that we
advertise for is cosmetology. It's 90% female. And then another one
is pick a mechanical engineering, or something like that, it's
predominantly male. Facebook, without doing any demographic
targeting on our end, Facebook will figure out to serve that
cosmetology ad to a female and that mechanical engineering ad to a
male within tens of impressions. You'll spend a few dollars and
Facebook will already have figured that out. And if, you can
imagine, that's kind of an AB test and you were running that you
might get a scenario where the cosmetology ad is doing really well.
You're showing it to 80% of the people and it's performing
well.
Rob Collie (01:24:10):
The other one is profitable, but it's just not performing as well
as the cosmetology ad. So typically, what people would do is they'd
call the cosmetology ad a winner and turn off the other one, which
is exactly the opposite of what you want to do. Facebook has
figured out the subset of people, the minority of people that
mechanical engineering ad is best suited for. And that's why you
want to leave it running, to show those folks that ad, even though
it looks like it lost to the cosmetology ad.
Kai Hankinson (01:24:41):
Yeah. And this is a very difficult thing to internalize. It's a
very difficult thing to build into your intuition. And I can speak
to this from my own experience. How many times have I come to you
and said, "Hey, this thing we're doing is producing a better return
than this thing we're doing. Let's turn off the thing that's doing
less well."
Brad Miller (01:25:03):
Yeah.
Kai Hankinson (01:25:04):
And pour more money into the thing that's doing better.
Brad Miller (01:25:07):
Yeah. And the one caveat to that is if you are budget constrained
and you can only spend a certain amount of budget, by all means,
send it to the better one. But assuming you're perfectly fine
spending as long as the spend is profitable and you don't have
those budget constraints, then you want to leave both going.
Kai Hankinson (01:25:27):
Yeah. If you live in California and you work in an internet
startup, you're not constrained by budget. The rest of us, out in
the middle of the country, little more constrained by reality.
Budget constrained but you guys don't worry about budget
that's-
Brad Miller (01:25:43):
Well, I mean, if it's profitable spend, that's the last thing you
want to be constrained by. It kills us every time we have a
profitable opportunity that we can't throw money at.
Kai Hankinson (01:25:54):
By the way, folks, you might as well just know Brad never lets me
turn anything off. If it's the equivalent of setting a pile of
money on fire in the backyard, Brad's like, "Uh-uh, no, keep going.
It only gets better from here."
Brad Miller (01:26:18):
That's how you make a million ad groups.
Rob Collie (01:26:18):
Yeah. So, that's the Facebook side. So, it's very much "Okay, come
up with some new ads." But other than that, it's set it and forget
it and let Facebook do its thing. Google, on the other hand, has a
million knobs that you could literally spend a lifetime turning.
And so, while it's AI is also super powerful, it's just not
implemented in the same way that Facebook is because that person is
giving you that intent. That person is giving you the signal of
saying, "I am looking for cosmetology classes." Whereas you're not
getting that signal from the Facebook user.
Kai Hankinson (01:26:57):
Intent makes it more efficient in a lot of ways. But now, like you
say, it's so much more tuneable. Even just the choice of keywords,
search strings that should trigger your ad versus not. Which of
these actually signals the intent to go back to school as a search
string? Probably doesn't mean I want to go to cosmetology school.
Probably means what are the back to school sales for my kids and
things like that. But how do you know? I don't know. You've got to
try it.
Brad Miller (01:27:32):
Yep. Exactly. Google is not going to get... You're not going to see
any statistical significance at Google. that's one of the
difficulties, in that if you have 17 million keywords a tiny
fraction of those are going to be spending at the levels where
you're going to get any kind of statistical significance to say
that that is profitable or not profitable, which is the
challenge.
Rob Collie (01:27:56):
Yeah. And so, you've determined that casting a wider net is a good
thing. The novice approach to this would be to come in and say,
look, "I know, naively, hubristically, I know the five types of
searches that people are going to run that I want to advertise on.
Going back to school, getting reeducated or something like that. If
you think about it naively, you'd think that the number of
keywords, essentially, is closer to five than 17 million. I think
that's something really, really interesting about the way you
operate, that you can even get to 17 million.
Brad Miller (01:28:34):
Yeah.
Rob Collie (01:28:35):
But also that there's ROI in doing so.
Brad Miller (01:28:38):
Yeah.
Rob Collie (01:28:38):
You guys wouldn't do it, if it wasn't ROI positive,
Brad Miller (01:28:41):
That's absolutely correct. And also we're forced in doing that
because in your example, let's just say round numbers. Let's just
say we get paid $30 for every lead that we generate. Some of the
best keywords in our industry, things like online colleges. Okay?
You can't get at the top of the Google search for less than $50 a
click. So if I bought a click for $50 and I'm only getting $30 on a
conversion that math is just not going to back out. So the only way
to compete is in the long tail. And so, we literally can't compete
at a short tail.
Rob Collie (01:29:20):
And how do you do that? Without giving away any secret sauce, how
do you get into that long tail business?
Brad Miller (01:29:28):
Well, the key at Google is making sure that your ad copy, your
search term or keyword, and your landing page are all in alignment.
The most important thing for Google is that their users to have a
quality experience, they want their ads to be as much like an
organic result as possible. So they have what's called a quality
score and they will determine your quality score for each of your
keywords based on how closely aligned your ad copy, your keyword,
and your landing page are. And they will penalize you heavily when
those are not aligned. And then that gets fed into you how much
you're charged per click.
Brad Miller (01:30:13):
So it is their way of basically incentivizing that alignment. And
so if you're able to take a long tail keyword or search term and
come up with finally tuned ad copy and a landing page that aligns
perfectly, you're going to get rewarded for that. The trick is how
do you do that 17 million times? So it's a combination. And going
full circle, taking us back to the poker business, it's a
combination of what can I do, programmatically, what can I use the
computer for, and what can I use the human for? And so it's
figuring out how to draw the line and have the balance between the
two where you can get scale, and yet at the same time to get that
bump in quality score.
Kai Hankinson (01:31:01):
Google's investment in the quality of the searcher's experience is
enormous to their credit. The quality score varies from one to 10
and they multiply your bid, when it gets inserted into the auction,
by that score. So somebody with a quality score of one is getting
their bid multiplied by one and someone with a quality score of
ten's getting their bid multiplied by 10. Now that means that
Google's... Imagine that if we go up against someone that's a one
and we're a 10. Let's say we bid a dime a click, or something, and
we end up paying a dollar and we win it. And they bid 95 cents and
lost. Well, look at what Google just did. Google just sold
something for a dime that they could have sold for 95 cents. And
that's an extreme example, but that's how wide the quality is
score...
Kai Hankinson (01:31:53):
That's how massive the impact is. And I think that's... Plus
quality score is a black box, much like the search engine
algorithms because they don't want anyone to be able to game it,
but that's why I think it is really important to not take shortcuts
and just try to build the entire business and website experience,
advertising experience around giving users what they actually want,
because that's what Google wants.
Rob Collie (01:32:20):
It's almost... You listen to it that way, they're willing to give
up almost like a 10X revenue opportunity in order to provide a
better experience for their users. It almost makes them sound
egalitarian, almost like they have our best interests at heart. You
could really talk yourself into that, couldn't you?
Kai Hankinson (01:32:38):
They have their long term best interests at heart.
Rob Collie (01:32:43):
Honestly, I think that's the best we can hope for, really, from
anything is everyone having a long term view. That's probably the
closest we're going to get to living at peace with each other is if
everyone had a long timeframe. I don't care about what happens to
the quality of the air. I'm not going to be around a hundred years
from now when it's really bad or whatever. Eventually everything
comes around to affect everybody. Hey, I'll take a company, that's
got its long term interest and view really clearly at the front of
its algorithms over the others.
Brad Miller (01:33:21):
Yeah. Makes sense.
Rob Collie (01:33:22):
I just talked myself into Google as a good guy. What do you know?
You were here when it happened. To get a little commercial for a
moment, because we don't run ads on this podcast so we might as
well advertise for ourselves.
Rob Collie (01:33:35):
So one of the things that we're inching closer towards is
combining. We'd be like Hank and Miller, P3 joint offering,
bringing the power of our consulting team and our analytics
expertise and our ability to span those silos, to provide the kinds
of feedback. With you guys, that have basically become like cyborgs
that are merged with these advertising platforms, and all the tools
that you have behind the scenes as well. Coming soon, whatever soon
is. This is one of those things, we're not in any huge hurry. We're
going to do this when we're ready, not sooner, but when we're
ready. We all believe in it. We're all proud of it. We're going to
start offering our joint services to others, help them navigate
this world of advertising and the same way that you guys have
learned to.
Rob Collie (01:34:28):
I think that's going to be a really valuable offering for the
world. If you look on the Power BI templates gallery, there's an ad
words template. It's just so cute. That's not going to solve your
problem. It's not even going to come close to solving your problem.
It ignores all of your own data on your side, like the things that
are happening on your end that the ad platform can't see. It
ignores all of this incredibly important tribal knowledge that
you've developed over the years like, again, all these
counterintuitive things. This ability to go after the long tail,
which is like even if you see the opportunity, which most people
wouldn't... Even if you see it, it's easier said than done.
Kai Hankinson (01:35:11):
Yeah.
Rob Collie (01:35:11):
I'm really excited about it. And, like I said, any opportunity that
we have to really hang our friendship out over a cliff, put money
in professional fortune stake we got to do that. It's gone so well
for so long.
Brad Miller (01:35:30):
It's time.
Rob Collie (01:35:34):
Yeah. In closing, when's the next email that's encouraging me to go
to Burning Man? I've added a recurring nightmare to my palette,
which is that I'm packing for Burning Man and I don't have all my
stuff and I got to show up and it is... It takes a lot to add a
recurring nightmare to your pantheon. And I've got one now. So how
long do I have before we start talking about this again? And I have
to start getting terrified and excited at the same time?
Brad Miller (01:36:04):
Well, Kai and I have been four times now. Our CTO has been, I
believe, 17 or 18 times. They canceled it this year, obviously,
because of COVID. So if they bring it back next year it is going to
be quite the event. It's also my wife's 40th and she loves it more
than anything else. So if it does indeed happen next year, we'll
definitely be there and-
Kai Hankinson (01:36:30):
See you at the Thunder Dome at sunset.
Brad Miller (01:36:32):
Yeah.
Rob Collie (01:36:33):
We've got some things to settle in the dome, don't we, Kai? Yeah.
You're right. Them canceling this year is like the whole Burner
community has just been like coiling a spring-
Kai Hankinson (01:36:48):
Yeah.
Rob Collie (01:36:48):
For an additional year.
Kai Hankinson (01:36:50):
Yeah.
Rob Collie (01:36:50):
Maybe next year is a good one to avoid. I don't know,
Brad Miller (01:36:55):
Probably, but there's no way I'm getting out of it. So I'll be
dragging you guys with me.
Rob Collie (01:37:00):
Like you say, most guys, when they're the wife goes out of town,
that's the time to party. When your wife goes out of town it's when
you get a rest.
Brad Miller (01:37:07):
That's right.
Rob Collie (01:37:11):
That's a keeper. So, Hey guys, we talked for a long time. I really
appreciate you guys devoting so much of your valuable time to this
and I hope the listeners do as well. So thanks again.
Brad Miller (01:37:26):
Thanks for having us.
Kai Hankinson (01:37:26):
Love you, Rob, take care.
Rob Collie (01:37:26):
Aw.
Announcer (01:37:26):
Thanks for listening to the Raw Data By P3 podcast. Find out what
the experts at P3 can do for your business. Go to
powerpivotpro.com. Interested in becoming a guest on the show.
Email Luke P- L-U-K-E-P@powerpro.com. Have a day to day!