Google and Facebook are immensely powerful, and are immensely rich because they sell huge amounts of advertising. Obviously, their ad revenue can’t be due in any substantial amount to a mass delusion that advertising on Google and Facebook works. It just can’t.
Here’s part of the immensely long transcript of a Freakonomics podcast with Dubner and Levitt in which they get various B-School academics to unload their suspicions that corporate America tends to spend way too much on advertising, both online and on TV:
Does Advertising Actually Work? (Part 2: Digital) (Ep. 441)
November 25, 2020 @ 11:00pm
by Stephen J. DubnerGoogle and Facebook are worth a combined $2 trillion, with the vast majority of their revenue coming from advertising. In our previous episode, we learned that TV advertising is much less effective than the industry says. Is digital any better? Some say yes, some say no — and some say we’re in a full-blown digital-ad bubble.
In our previous episode, we learned that more than $250 billion a year is spent in the U.S. on advertising. Globally, the figure is nearly $600 billion. That’s more than half a trillion dollars. On advertising. Because of the digital revolution, television advertising has lost some of its primacy. But TV still accounts for roughly a third of ad spending in the U.S.; the Super Bowl alone brings in more than $300 million.
And how effective is all that TV advertising? I mean, how good is it at actually selling the products it’s telling you to buy? The conventional wisdom says it’s got to be effective; why else would companies spend so much money on it? But the data tell a different story. Here’s what we heard last week from Anna Tuchman. She’s a marketing professor at Northwestern University, and she recently co-authored a massive study on the efficacy of TV advertising:
Anna TUCHMAN: This means that doubling the amount of advertising would lead to about a 1 percent increase in sales.
Stephen DUBNER: So, your research argues that TV advertising is about 15 to 20 times less effective than the conventional wisdom says, yes?
TUCHMAN: That’s right.
There are, not surprisingly, objections to this research. Especially from the marketing industry. For instance, they’ll point to the brand-building aspect of advertising: “It’s not just about short-term sales,” they’ll say. Or the game-theory aspect — that is, if you don’t advertise your product and your rivals do, where does that leave you? Still, any company that spends even thousands of dollars on TV ads, much less millions or billions, would have to be sobered by Anna Tuchman’s findings.
Was TV advertising always so inefficient — or did it lose its luster recently with the arrival of digital giants like Google and Facebook? We don’t know the answer to that question.
I’ve often recounted the curious tale of how in the 1980s I worked for a marketing research start-up that created perhaps the all-time best real world lab for carrying out Randomized Controlled Trials.
In eight towns, we bought the new laser beam checkout scanners for all the supermarkets and drug stores in town in return for their cooperation. We recruited 3000 households in each town who agreed to identify themselves to the checkout clerk, so we could record all their consumer packaged goods shopping. And we controlled what TV commercials they saw on cable TV. So we could divide our sample into test and control groups that had exactly identical amounts of purchasing of the client’s product over the previous year and then show them more or different ads for a year and measure how much their purchasing increased.
Even today, this sounds like science fiction, but it was all up and running 40 years ago.
Brand managers at CPG firms were initially wildly enthusiastic: finally, they could scientifically prove to the bean counters at headquarters that their advertising is so effective that they should double their ad budgets!
But after a half decade of spectacular growth in our service, it turned out more or new advertising only rarely increased sales. And I don’t mean that doubling the ad budget only increased sales in the test group of thousands of families by, say, 7% over sales in the control group of families, and that didn’t quite payout in terms of profitability. No, I mean, the typical result was that the sales in the test cell that saw twice as many ads as the control cell bought 0.1% less.
I worked on two internal meta-analyses of our testing results. Sometimes, more or better advertising moved the needle. For example, I managed in the field the rollout of a new brand extension that was chemically new and improved to a significant extent. The advertising explained that there was a problem that you didn’t know you have but now there’s a solution for it. Not surprisingly, that complex and useful message did better with more TV commercials explaining it.
But most of the time, the clients didn’t have new news like that to convey. It was just the same old product they’d been advertising on TV since 1947, so showing more ads did zip for sales.
My guess is that when CPG firms started advertising on TV in the late 1940s they were frequently bringing out innovative new consumer packaged goods that benefited from massive television advertising explaining why this breakthrough product would make your life better (which it would—the life of the American housewife improved rapidly in the postwar era, in part due to much better products. Everybody today makes fun of old black and white TV ads in which housewives are ecstatic about their new laundry detergent or whatever, but the postwar actually was a golden age of reduction in the ancient drudgery of keeping a home).
For example, the CEO of my client had been a junior brand manager in the 1950s when he came up with a plan:
It worked.
But by the 1980s, innovations of that magnitude were harder to come by, so a lot of CPG advertising was then aimed less often at informing you why it would be in your self-interest to buy this new and improved product and more to persuade you to keep buying the same old thing.
Nor did hot new ads that the ad agency had guaranteed would excite the public move the sales needle very often. My vague recollection of our tests is that perhaps one ad starring Bill Cosby sold more product—Americans loved Bill Cosby in 1983. But I may have even that wrong.
But after about a half dozen years of brand managers’ hopes being routinely crushed by our scientific experiments, clients started losing interest in our superb testing service, and it was eventually shut down.
Before then, I had suggested to my client, a very big league marketer of famous consumer packaged goods brands, that they do more tests of cutting their advertising, as they had done in one test with zero harm to sales. But they replied that their firm believed in television advertising. (Also, no brand manager had ever become CEO by cutting his ad spending. You climbed the corporate ladder by convincing the big bosses to give you a bigger ad budget.)
… The internet has made it almost too easy to sell to us. And sell to us they do. Last year, advertisers spent $123 billion on internet ads in the U.S., just less than half the total ad spending across all media. That’s how Facebook and Alphabet, the parent company of Google, have become two of the most valuable companies in the world. More than 80 percent of Google’s revenue comes from advertising; more than 98 percent of Facebook’s revenue comes from advertising. With so many advertisers spending so many billions, they must be getting a healthy return on their investment, right? So, digital advertising must be effective, right?
Steve TADELIS: Uh.
Tim HWANG: Um.
TADELIS: Uh.
HWANG: “Oh, ads definitely work. But we can’t tell you how or why or give you any evidence for it.”
Dubner goes on to talk to some B-school academics about how it’s hard to get a straight answer out of anybody on this giant question.
My guess is that because it’s super easy to do the kind of A/B testing on the Internet that we could do with TV ads in the 1980s that digital advertisers must know their ads work. Right? Right?
By the way, back in 1996 the Chief Operating Officer of our company was quitting to become a Silicon Valley start-up founder, but he didn’t have an idea what to found. In his negotiations with the firm, he got the right to take one employee with him, so he chose me. He asked me what we should do, so I told him we should do A/B testing on the Internet. We know more about how to do A/B testing on TV than anybody else in the world, so we’ll figure out how to do it on the Internet.
But then I got cancer and couldn’t quit and make my fortune. Oh, well …
… Tadelis took his concerns back to his bosses at eBay. He proposed a different way to understand the impact of the online ads eBay was buying: he offered to run some randomized experiments. To a researcher, that’s the gold standard.
TADELIS: And there was not any buy-in.
Eventually, they got eBay to do some RCTs:
They turned off all their keyword-search ads, then measured actual sales:
TADELIS: And the impact on average was pretty much zero.
What was eBay’s existing belief about paid-search advertising?
TADELIS: The company believed that roughly 5 percent of sales were driven by paid-search advertising, meaning that they believed that if we would pull the plug on advertising, sales would drop by 5 percent. What we found was that sales dropped by about half a percent. So, that’s an order of magnitude less. And it was not statistically different from zero.
But maybe it’s still worth it to gain even that half a percent? Now we have to know what the advertising costs, and measure the return on investment.
TADELIS: When you did the return on investment for every dollar that eBay spends — eBay believed that for every dollar they’re spending, they’re getting roughly a dollar-and-a-half back, meaning 50 cents of net profits. And what we showed is that on average, they’re losing more than 60 cents on every dollar.
So, how did these results go over?
TADELIS: Well, the president of eBay, who later became the C.E.O., he cut the paid-search marketing budget immediately by $100 million a year.
So, what happened next? You might think — what with capitalism being the hyper-competitive, market-optimizing, perfect-information ecosystem it’s supposed to be — you might think that other companies, once they learned about this eBay research, would cut their online ad spending. Or at least commission their own research to test the theories. So, did they?
TADELIS: Excellent question. There was a lot of chatter online after our experiments became public, suggesting that folks at eBay don’t know what they’re doing. And paid-search advertising works wonderfully if you know how to do it. But of course, that was backed with no data and no analysis.
In other words, the digital-ad community did not rush to replicate the results. Now, given the opportunity to save millions of dollars that the eBay research showed was being wasted, why wouldn’t other companies at least poke their own data a little harder?
TADELIS: Well, I think there are many reasons. Let’s start with the way in which this industry is structured. You could think of four different actors here. There’s the customer, which is the company or the person who wants to advertise in order to get business. And then you have three players sitting on the other side of this market. One is the publishers. That would be Google. That would be The New York Times, or any other place where the ad appears in front of people. The other are the people who create ads. And then finally a smaller part of the industry are these analytics companies that, like that company eBay hired, are trying to help companies spend this money. And if you think of all these three players on the other side of the fence, no one there has an incentive to basically open this Pandora’s box.
Even within the company that’s buying the ads, the incentives can be complicated. Steve Levitt again:
LEVITT: If you think about it, no chief marketing officer is ever going to say, “Hey, I don’t know, maybe ads don’t work. Let’s just not do them and see what happens.” So, don’t get me wrong. I’m not implying that advertising doesn’t work. I’m implying that we don’t have a very good idea about how well it works.
Steve Tadelis agrees. The potential for digital advertising would seem especially large, given its ability to micro-target consumers.
TADELIS: And targeting really is key because one of the lessons we learned from the experiments at eBay was that people who never shopped on eBay, they were very much influenced by having eBay ads for non-brand keywords. You know, “guitar, “chair,” “studio microphone.”
So if you can find somebody who is looking to buy a guitar but who doesn’t normally think about looking on eBay, then you’ve hit marketing gold. At least in theory.
… HWANG: A few years back, Procter & Gamble, which is one of the largest advertisers in the world, decided that they would run a little experiment. They were going to take about $200 million of their digital-ad spending and just cut it out of their budget to see what happened.
Procter & Gamble said they were doing this because of concerns over brand safety and the proliferation of bots, which can pollute the data on ad impressions.
HWANG: And the end result was fascinating. Basically, they said that there was no noticeable impact on their bottom line.
Again, the ad industry will have a lot of explanations for why this might be. Or for why there’s a lot of value in advertising beyond short-term sales figures. But Procter & Gamble is a big player. Even if they’re wrong, even to a small degree — they’re the ones whose money drives the advertising ecosystem. What would happen if this turned into a mass movement among advertisers? One shouldn’t underestimate the size and reach of the advertising ecosystem. The sports you watch on TV: supported by ads. The journalism you consume: supported by ads, at least much of it. Google Maps and Google Drive and — well, Google: supported by ads. As well as Facebook and Instagram and Twitter and nearly everything else you consume online and don’t pay for. Including this podcast and just about every other podcast you listen to.
This was back during the Cola Wars of the 1980s, which led to the New Coke fiasco and Michael Jackson’s hair on fire.
It’s possible that our methodology wasn’t good at measuring what really matters to advertisers like Coke and Pepsi, which is persuading impressionable young people to decide, “I’m a Pepsi person” or whatever and thus become a lifelong loyalist. Our main panelists were typically the mother in the family. Maybe what really matters is getting a 16 year old girl to decide that she, unlike her mom the Pepsi buyer, is a Coke drinker?
Warren Buffett made a fortune buying Coke stock because of the company’s huge brand equity from 90 years of advertising: even if Coke wasn’t making a lot of money right now, somebody would come along and figure out how to make more money off the Coke brand, which eventually happened.
Anyway, it’s all very curious. After all these decades, I still really don’t know what to think.
