Before you read the rest of this post, go to Google and try searching for “Amazon.” You’ll probably notice that the top two listings are both for Amazon’s website, with the first appearing on a light beige background. If you click on the first — a paid search ad — Amazon will pay Google for attracting your business. If you click on the second, Amazon gets your business but Google gets nothing. Try “Macys,” “Walgreens,” and “Sports Authority” — you’ll see the same thing.
If you search for eBay, though, you’ll find only a single listing — an unpaid one. Odds are, after marketers at Amazon, Walgreens and elsewhere catch wind of a preliminary study released on Friday, their search listings will start to look a lot more like eBay’s. The study — by eBay Research Labs economists Thomas Blake, Chris Noskos, and Steve Tadelis — analyzed eBay sales after shutting down purchases of search ads on Google and elsewhere, while maintaining a control set of regions where search ads continued unchanged. Their findings suggest that many paid ads generate virtually no increase in sales, and even for ones that do, the sales benefits are far eclipsed by the cost of the ads themselves.
Companies spend enormous sums on marketing their products. Yet it’s notoriously difficult to measure the impact of ad expenditures. Companies advertise heavily at times when they hope to sell a lot — like Christmas Eve and Boxing Day — and in areas where they expect to see their sales grow. So a naïve examination of the relationship between ad expenditures and revenues will of course find they move in sync, even if customers don’t pay the ads any mind.
Advertising has also traditionally produced a lot of waste — I see ads for Brioni suits when I open up the morning paper, even though the last time I wore a suit was on my wedding day. The study’s authors quote 19th century retailer John Wannamaker: “I know half the money I spend on advertising is wasted, but I can never find out which half.”
The internet promised to change all that. Google, Yahoo, Bing, and others gave sellers the opportunity to target their pitches to customers who were plausibly interested in their products. That’s why paid ads for Amazon come up in response to a search for books, but not life insurance. Further aligning the interests of companies and consumers, advertisers only get charged for paid search listings that actually get clicked on, ensuring that they pay for attracting genuinely interested customers.
But what do companies actually get for the billions they now spend on search marketing? The eBay team began by examining whether there’s any benefit to buying search ads that contain the word “ebay.” In these cases, it’s possible that in the absence of paid listings, customers would simply click on the unpaid — or “natural” — listing, which would appear at the top of the search anyway. So in March 2012, eBay conducted a controlled trial to see what would happen if they shut off this “branded keyword advertising” by halting their purchases of search ads containing the word “ebay” on Microsoft and Yahoo search engines, while continuing to purchase search ads on Google as a control. There was no change in eBay sales via Yahoo and Bing, relative to those that came through Google — consumers simply substituted clicks on the unpaid search listing for the now-absent paid ones.
Encouraged by these findings, eBay management agreed to run a controlled experiment where they shut off all Google search ads in a third of the country, while continuing to buy ads everywhere else. In contrast to branded keywords — where it’s inevitable that the company will end up as one of the top unpaid listings — there’s a good chance that if you try searching for “used les paul guitar,” a guitar reseller will appear ahead of eBay’s search listing. So in order to drive a customer to eBay for his guitar purchase rather than, say, Guitar Center, it might be worth the cost of placing a carefully targeted ad.
But in aggregate, that’s not what the eBay team found — overall, there was no appreciable decline in sales of eBay listings in the part of the country where Google ad purchases were shut off. People who thought to buy guitars via eBay were finding their way to the site anyway, either by clicking on natural listings, or by going directly to eBay’s site without using a search engine at all. Search ads did generate a modest increase in the likelihood that internet surfers with little recent history of eBay transactions would end up making purchases on eBay. So paid search ads serve an informational function, letting a sliver of potential eBay customers know that they’re in the guitar business. But by the time you get to customers who have had three prior eBay transactions in the last year, the effect of paid search on sales drops almost to zero. Overall, paid search turns out to be a very expensive way of attracting new business: The study’s authors estimate that, at least in the short-run, paid ads generate only about 25 cents in extra revenues for each dollar of ad expenditures. (For branded keyword searches, the additional revenues are close to zero.)
People buying search ads aren’t idiots — they’ve looked at the correlation between keyword purchases and subsequent sales and no doubt found it to be strong. But this study suggests that marketing departments should be more careful in confusing causation and correlation in assessing the returns to their ad expenditures, to avoid the equivalent of concluding that marketing works because you advertise and sell a lot in December.
The study’s authors note that paid search may be more profitable for other companies than it’s been for eBay. For example, as Stop and Shop tries to get a foothold in the crowded New York online grocery marketplace, they might sensibly buy some ads to compete with Fresh Direct. Paid search may also be worth it for smaller companies that lack the name recognition and high Google page rank that make paid searches less valuable for the eBays and Amazons of the world — some of Google’s own research indicates that this is likely the case. Caveats aside, eBay’s experiences suggest that all companies should look carefully at how much bang they’re getting for their search marketing dollars.
The larger lesson from eBay’s experiment is about the importance of questioning conventional marketing wisdom. As much as the internet has given companies opportunities to target their ads, it’s also given them a ready testing ground to experiment with different business practices to see what really works.
