Blog Measurement Needs Standards

New data from Gallup notes that one in five Americans say they consult blogs “frequently” or at least “occasionally.” Earlier Gallup figures found that 21 percent of 18-29 year olds read blogs, while Pew pegged the blog consumption audience at 58 percent last year.

Here in lies the problem. It's impossible to say if blog readership is trending up or down because everyone measures it differently. The research community needs to set some standards for measuring in aggregate how many people read blogs, listen to podcasts and subscribe to RSS feeds.

More importantly, it would be helpful if an organization or group of organizations got together to set standards for how a blog will be measured against others in its category, much as the ABC does for magazines and newspapers. Perhaps Nielsen BuzzMetrics will play a role in leading such an initiative.

Finally, I remain unconvinced that traffic data or unique users is the benchmark we will use to size up blogs. This is short thinking in a Long Tail world of content. Some standardized metrics (TBD) are going to become essential as the battle between the various blog advertising/publishing networks heats up and, with it, investments in blog advertising. Who's going to set some standards?

Technorati Tags: Advertising, Blogs, Gallup, Long Tail, Measurement, Pew, Stats

Comments

[Jay Small]

Steve,

I know you realize what a challenge it will be to define and socialize standards for measuring blog success.

At this point, I'm not at all sure we even have standards for how to define "blog." But let's say we get that right, eventually.

Every possible measure of a Web site's success is readily prone to (a) fraud, (b) spam, (c) unacceptable margin of error or (d) all of the above.

Most analysts agree on the basics of measuring page views, at least. But different sites use different packages that may or may not distinguish human-generated views from bots.

"Uniques"? To any attempt to standardize, I simply say, "Ha!"

Inbound links? Trackbacks? Number of comments? All prone to spam.

Third-party, panel-based ratings? Misleading. Generally, the sample sizes are too small to be statistically valid.

I measure the success of my own site by purely selfish terms:

1. Does it showcase what I do and can do to the people I want to see it? For the most part, yes.

2. Do I get business inquiries and referrals from it? Yes.

3. Are my face and name known and celebrated in every pub on every streetcorner of every city in America? Um, no, but hey, a guy has to have dreams.

[Max Kalehoff]

Steve,

You're right...we do need standards and metrics, and our new industry is making progress!

One of the most important initiatives is being led by the Word of Mouth Marketing Association (www.womma.org), a trade group which Nielsen BuzzMetrics co-founded in 2004. In fact, setting standards to encourage use of word of mouth and consumer-generated media (like blogs and other platforms) is one of WOMMA's three primary goals. The WOMMA Standards & Metrics council is co-chaired by Jonathan Carson, CEO of the Nielsen BuzzMetics service, and much further info can be found here: http://www.womma.org/research.htm.

In addition to broad industry efforts, Nielsen BuzzMetrics is working hard as a research and technology innovator to keep raising the bar in a constantly evolving CGM landscape. Much of our industry's understanding and applications relate to textual media, which comprise the vast majority of consumer content creation today. But it's no secret that democrotized publishing capabilities are spawning a new era characterised by what my new colleague, Pete Blackshaw, has coined consumer-generated multimedia, or CGM2. Stay tuned. That includes not only text, but audio, video and other formats. No doubt, these are exciting times. There are exciting developments ahead.

[Mike Driehorst]

Like other media, blogs will be measured by who sees them them:
> Visitors (not only comments posted)
> Links from other blogs

Like other media, it's all about getting eyes.

Now, the value of what is read on a blog depends on its audience. I posted just last week that, depending on the blog (its targeted audience or niche), a post about a product, company, person, etc., can be more valuable than a similar article in traditional media.

I'd definitely would welcome thoughts from you Steve, as well as from anyone else.
http://www.mikespoints.com/archives/001273.html
Take care,
Mike

[Brent]

I agree with Mike. I love the ideas of standardized metrics, but how can you effectively quanitfy degrees of influence? Blog audiences are miniscule compared to other media (tv, magazines, etc.), but make up for that defecit by being hyper-targeted to specific audiences. Won't a generalized number miss those varying degrees of influence that make smaller blogs so valuable? A comparative number could work for the top 1 or 2 percent of blogs with massive audiences, but I think a number for anything smaller misses the point.

[Shel Holtz]

How many of those surveyed read blogs without recognizing that's what they are? Hell, only 4% of online Americans subscribe knowingly to RSS feeds, but 20-plus percent subscribe without knowing that's what they're doing. It seems to me the wrong question is being asked. Why not ask how important it is that a website provide the means to comment on content, for example, or to read comments submitted by other readers, or to follow links to other sites that reference content on the originating site? The results of such a survey might be significantly different.

[Kami Huyse]

David Parmet made a good point in the comments section of my blog that the word blog is often not even recognized by those who are surveyed, therby skewing the results. So we even have issues with the terminology.

[Luke Welling]

While I agree with your basic point that they are becoming important (and commercial) enough to need standard measuring techniques, your evidence is faulty.

You complain that a study in November 2004 put readership at 58% and a different more recent study put it at 20%.

Actually, the BBC 2004 report says readership is up by 58%, taking it to 27% of online Americans. The article says 120 million Americans were online in 2004, which is about 40% of the population. That means that study was saying 11% of the population had read a blog.

Even without measurement standards, the increase from 11% of the population "have read a blog" to 20% read blogs "frequently" or at least "occasionally" seems fairly consistent and believable over the space of a year or so.

[Jay Dwivedi]

I totally agree with you when you say "I remain unconvinced that traffic data or unique users is the benchmark we will use to size up blogs." We have gone through this before. In my opinion this metric is meaningful only for those bloggers who blog as a hobby. For everyone else, it should be net income generated as a measure of success.

[Lou]

I think evaluating a blog's success goes far beyond the raw numbers. In fact, the raw numbers (PVs, uniques, etc) miss the point almost completely unless your ONLY goal for the blog is more ad impressions to drive revenue.

I like Stowe Boyd's Conversational Index, and Don Dodge's mods to that formula, much better. And while these simple formula approaches don't completely answer the question of whether or not a blog is successful, I think they go a lot further than simple web analytics alone.

I am involved in blog projects for a media company where the primary goal is to engage the audience, and the secondary goal is to keep making that audience larger. A small audience that is engaged is better than a big audience that is unengaged and therefore tends to be transient and non-loyal. A small (at least to start) engaged audience can be grown by leveraging that engagement. Of course, the devil's in the details on that last statement.

[why]

1. WHY do we need to measure this?

2. The most important aspect of blogs is precisely what you cannot measure, it's ability to have an influence. One blog entry, read by just the right person, can change things mightely if that person is an opinion leader or has lots of power or resources. Similarly, a huge number of hits from people who do nothing with the information, means nothing.

In other words, it's the content, not the count.

[Katie Delahaye Paine]

Steve, this discussion will be the subject of a session I'm organizing at The Measurement Summit, organized by the Institute for PR, October 4-6 in Portsmouth, NH. We would love it if you'd join our conversation.