Crowdsourcing a New System for Measuring Influence (Beta)

When I started blogging a few years ago I quickly got to know Steve Gillmor. I love Steve. He consistently says things are dead - Office, TV and links, just to name a few. Sometimes Steve is right, other times he is way off. But he missed a big one.

The practice of measuring online influence by links is truly dead. Link authority, as it was called, was good while it lasted. When blogs were where all the action was, this system was king - even though many of us hated it. Good for Technorati for building its own brand around it. Nowadays, however, link authority is a meaningless metric. Jeff Jarvis touched upon the broader measurement issue last week.

The main reason link authority is dead is that there are so many places where people can publish and connect with peers. Often, many of us are active in more than one at a time. You might not think some of these networks are influential, but in fact they are.

Take Twitter, a small example. Lots of people who are on Twitter don't write blogs and vice versa. Still, it's influential. Thousands of people track their friends and the site is incredibly well optimized for search. The TV nets are checking it out too.

Last night on Twitter my friend Robert Scoble and I got into an intense discussion regarding Facebook. Arguably, some 6,000 people (Robert's 4,400+ followers and my 1,500) witnessed it. That's small compared to the reach of our blogs.

However, one of those folks was Dwight Silverman, a veteran tech reporter from the Houston Chronicle  He took it all in. (Dwight had to put up with my PR pitches from 1996-2004. Pity him.) Had Dwight written about this, then conceivably hundreds of thousands more would have been influenced. And link authority or impressions doesn't measure any of it.

With this in mind, lots of smart people in our firm have been thinking about online influence and how to measure it. This is a critical issue not just for PR pros, marketers/advertisers, but everyone who wants to monetize content. We're experimenting with a new weighted blended approach and would like your feedback. (When I learned that I ranked highly in their calculations I told David Brain, our Europe CEO, that he fixed this to psych me into relaxing so I Twitter less. My rankings would fall and his would rise!)

The model we have developed is far from perfect, but it's a start. The key is to develop a system that can grow as the channels change. We want a system that we all like - or at least a majority. Feel free to leave comments here or on David's site. Our intent is to create this in partnership with you out in the open. It's in beta but now we need your help. Together we can find something that's workable.

Comments

[Tinu Abayomi-Paul]

The thing that always concerned me about using links to measure popularity, even when most people think it was "right", was that the link system within Technorati doesn't seem to take into account the quality of the links pointing, rather than just the number. Also, only certain types of links are counted, and even then, only within their system.

In terms of potential reach, there are so many things to take into account, and any metric needs to take into account quality - I could reach thousands of people fairly quickly on LinkedIn if I had the time to execute a strategy... as I could on Facebook, but the type of people I'd reach are different. In this case, whatever the new consensus is on measuring influence, I hope it judges more by the quality and depth of penetration into a given demographic, rather than just by pure potential of audience size.

[Steven Hodson]

While I agree that the current metric being used is rapidly losing value I am not sure if what you are proposing will do anything but making it harder for those that aren't already a top influencer; and who hop on all the new stuff that comes out, make it to the top level.

Not to mention the fact that portion of the idea are open to being badly gamed.

http://www.winextra.com/2007/07/17/well-it-looks-like-sht-is-going-to-hit-the-bsphere-fan/

[Neil]

Great. Just my luck. Just when I start getting all these new links, the rules to the game changes.

[Reasonable Robinson]

Fascinating discussion. This issue applies to any 'idea' that has reach. I'm particularly interested in the reach and influence of management ideas...stalwarts such as 'the marketing concept' or fads such as BPR. There's some interesting philosophical hints in the area of Critical Realism that has alot to do with the 'power' and dominance of the individual or managing group. I reckon some sort of qualitatitve 'veracity' benchmark might be interesting. This of course then ties in with 'consensus' theories of 'truth'. This bascially means that authority/ influence is actually 'agreed' by the audience and is not some independent 'thing' that can measured objectively. Not unlike DIGG / Reddit voting. The essential issue for me here is that maybe to measure 'it' you need to think in terms of subjective measures and not the technical/ analytic type measures that dominate much of the benchmarking process in this area.

[Josh Bernoff]

Why only one metric, even a blended one?

I know you think links are dead but they do measure something.

The other metrics you have measure something else.

How about how many times your name is mentioned in a Google search?

One person might have more influence with professionals, another with the masses. Or one might have more influence long-term, another right now. Who is more famous, Paris Hilton or John Travolta? Depends what you're measuring.

Even though our blog ranked #25 on your final Social Media Index, I know your PR people contact us partly because of all of Charlene Li's and my press quotes, which aren't even on your list. Old media is supposed to be dead, but it seems to matter to Edelman and every other PR firm.

If you have more friends than me, but my friends include Walt Mossberg and Steve Tedeschi, who's got more influence? What if my friends were to include Sergei Brin and Rupert Murdoch (not that they do, but I'm just saying). Hey, I've got YOU on my list of friends, that has to count for something, too.

Reducing this to numbers is a nice idea, but there's something missing. Your colleagues in PR understand this, it's clear to me.

Still interesting, though. Keep working on it. . .

[Reasonable Robinson]

Absolutely concur with the last comment. Surely a belnd of hard and soft metrics will give a balanced picture. Pick any article or book by Tim Ambler for thoughts on this. I'll be posting my marketing metrics literature review in parts on http://gullibility.blogspot.com shortly for those who might be interested.

[David Weiner, PR Newswire]

Steve,

I agree with most of what you're saying but disagree about the usefulness of authority. When monitoring and measuring for clients, this metric is still vital ... even if only to separate the real hits from the blog spam. That is one inherent problem with the new technorati.

See ya soon!

DW

[Jason Stoddard]

Steve, you hit on a subject we think about a lot, as the marketing industry struggles to understand metrics beyond CPM, clicks, and CPS. Social Media Effectiveness is a great first step towards a new metric that will be meaningful and monetizable.

But (there's always a but, isn't there?) why stop at social media, and why stop at attention? We've been kicking around two metrics that we'd like to see made into reality, called Attention Index and Monetization Effectiveness, the first measuring the value of a particular individual's attention, and the latter measuring the bottom-line payoff from that attention. Yes, grandiose measures that perhaps only Google can deploy, but here are our thoughts:

http://www.centric.com/thought/2007/07/18/attention-index-and-monetization-effectiveness/

[Flemming Madsen]

I see at least 2 major problems with what you are proposing.
First of all, Influence is contextual. A person may have influence on the debate on blogging, but probably not on men’s fashion (just an example), climate change or accounting software.

Second, your conjecture doesn’t take indirect influence into account. This was acceptable when social and indirect influence was insignificant (when the head-count metrics were developed in the 50s), but with internet penetration indirect influence is often much bigger than “direct influence” (aka popularity). Steve, your influence is greater than the number of people who read your blog, but your metric doesn’t take this into account.

In essence what we are looking for is a metric that explains (or correlates very strongly with) outcome. How does your metric correlate with outcome? You say it isn’t perfect, but exactly how good is it? How do we know if the next iteration of it is better or worse? Any thoughts/analyses/research?

In my view we can learn a lot from other areas of science where network analysis, including the analysis of influence, has been used for decades.

[David S]

http://austria.phys.nd.edu/netwiki/index.php/Centrality

I'm by no means an expert on the subject, but links really only measure degree centrality, which is ultimately not as important as betweenness (aka _direct_ influence on other network nodes). Another key metric is closeness, which in this context would represent the _time_ it takes for a blogger's opinion to circulate to others, potentially as calculated on the basis of link timestamps. A more powerful degree centrality might measure more than a simple count of the number of links by incorporating eigenvectors.

Putting the above aside for a moment, there's potentially a big difference between influence and awareness. Influence in the real world implies a change in behavior, such as the purchase of a product. Because most bloggers or similar other nodes within a social network probably have different objectives (generate traffic to sell advertising, out-scoop competitors, promote their firm, you name it), a one-size-fits all approach might not work. If however you define influence as the ability to drive awareness in a broad sense, then it's a different story. But again, I think this is an important distinction to make and further research would be needed to establish the impact of the greater awareness on the desired end result.

[Dave]

Funny you should mention it, as I've been seeing "PocketTweets" and "Twitter" in my own referrer tracking more and more. Obviously, those visitors did not come via any weblink.

[Natalie]

Hi Steve,

If you need any help in this issue, just let me know.