I found two marketing articles quite thought-provoking this week:

• Ad Testing: Are You Using The Wrong Success Metrics? (Brad Geddes @ SearchEngineLand)
• Which Metric Would You Measure if You Could Choose Only One? (Erez Barak @ SearchEngineWatch)

The common factor, of course, is “which metrics should marketers use to evaluate performance?”

Brad Geddes’ article is interesting. He identifies the “usual” key metrics, including both intermediate and “final” measures. But he then jumps off a cliff by proposing a new measurement (Profit Per Impression) that foolishly incorporates “impressions” in the final measure. I think he’s confused “relevant” with “most important.” I’ll analyze his examples further below.

Erez Barak’s article was actually more thought-provoking, for me, because he explains how and why different people, in different roles in a company, view different metrics as “most imporant.” Perhaps I’m biased after focusing the past 15 years of my life on ROI-based marketing, which might explain why I was surprised to consider the possibility that there are rational reasons why people in other roles might view other measures as more important. Those perspectives are important to understand, even if I conclude (as I always do) that ROI is the most important metric.

Which brings me back to Brad Geddes’ article.

First, his article is a bit confusing because of the data examples: I think the first table shown in the article isn’t the one the author intended, and it’s unclear whether the green background highlighting (for the “best” value in each column) was included to lead (or mislead) the audience. Here’s the raw data I think he intended to show (but without some of the right-most columns):

He asked a group of marketers, “Which of these ads performed best?”

Ads 2 and 5 were each selected by about one-third, while Ad 4 was selected by 20%. Another 7% concluded that “there wasn’t enough data to determine a winner” (probably due to the absence of some of the right-most columns, or perhaps the statistical validity of the data sample size).

Important Note: the “profit” column doesn’t represent actual profit, but merely subtracts the cost of advertising from total revenue (ignoring cost of goods and other expenses). Apart from being grossly misleading, this also ignores the likely differences in actual profitability (unless margin is identical for all products). I’m assuming that the data is shown for a single keyword-phrase on AdWords.

Geddes starts by stating that “not enough data” was the correct answer, if only because he didn’t provide data for “profit per impression” (PPI).

Once that data is revealed, Geddes argues that the best performance came from Ad 1, because it delivered the best “profit per impression” (PPI).

Geddes’ argument: “Yes, every single impression has an opportunity to generate a sale. Therefore, your measurements should start at the impression as well.”  While he’s correct that “impressions” is an important measurement, it’s not a critical metric for advertisers.

To me, the “most important metric” is ROI (measured as profit per dollar spent). Here, the second ad draws ROI of 1929%, which means that for every dollar spent, the company earns $19.29 in profit.  I’m not going to suggest that the merchant instead use an ad that draws only $15.61 in profit per dollar spent, especially when that ad generated less total profit than the ad with higher ROI.

Other metrics are important, of course. Neither my chart nor Geddes’ includes “cost per impression,” even though that’s the key metric used by Google, which uses its prediction of that value (their “revenue per impression”) as the main factor in choosing which ads to show, and in what position, for a particular keyword-phrase.  (Another important factor, Quality Score, isn’t provided for these ads.)

For me, ROI is the primary metric to use. But of course, this brings us back to Erez Barak’s article, which acknowledges that as a marketer, that’s the main metric he’d expect from me — but that there are valid reasons for people in other roles to prefer different metrics. And in that light, I certainly acknowledge that “profit per impression” is a potentially useful metric, even though I don’t agree that it’s the most important.

Comments are closed