…and anyone else in the blogosphere who thinks that they are the only ones thinking about how to measure “…social and niche media online.”

Have you heard of web analytics?

Do you know that there is a whole community of talented and driven people trying to figure this out?

Have you heard of the Web Analytics Association?

Did you know that there is a conference dedicated to measurement on the web?

I was mildly exasperated last fall when Scoble complained about this, Jeremiah Owyang picked up on it and has done some nice work trying to facilitate the discussion but now I’m in a full-on tizzy.

Ok Jeff, now that I’m done ranting, let’s take a look at your specific points, knock my head against them and see what comes out.

You Say, “Except I argue that the panel means of measurement is doomed to miserable failure in the mass of niches.”

That may be true. So what? Panel-based measurement is not about measuring the fringe. In fact, the measurement companies don’t try to hide this. They are very careful and transparent about which data they have confidence in and which data they don’t have confidence in and statistical measures based on a low sample size have low confidence. Looking at it from the other side (I used to work for a relatively large publisher), services like comScore and Nielsen and Hitwise are the common language that publishers and advertisers use to work together and it seems to me (please show me if I’m wrong) that advertising dollars, by and large, still go to where the majority of the traffic is. So, the big players like Yahoo! and MSN (and even some not so big players) are still served pretty well by panel-based measurement.

Maybe you’re not really upset over these companies’ inability to measure the fringe, which is symptomatic of the market in which they operate, but rather you are rallying for a redistribution of the advertising wealth? Dunno, just playing random association here.

You Say, “What we need, I’ve long argued, is standard metrics reported from the sites’ servers or from snoopers on page and verified by a service such as Comscore or Nielsen. Old methods will not work in this new world. The same goes for Nielsen, which is buying the rest of NetRatings.”

My basic reaction to this is, “Do you remember ‘hits’?” Hits (server requests) are a standard metric reported from the sites’ servers and they were grossly abused. Thus, the page view (yes, we’ll get to that statement in a minute). There are companies that will audit and validate (ABCi comes to mind) site-analytics implementations but I’m not aware of any large scale adoption of this methodology in the online space as an effective tool for getting publishers and advertisers together.

Anecdote: Thanks to Katherine Paine we number nerds web analysts have an amusing acronym: H.ow I.diots T.rack S.uccess

You Say, “And while we’re at it, let’s figure out the new measurements that capture the unique value of this new medium: authority, speed, connectedness… The page view is dead.”

First of all, discussions of how to measure this new medium (I’ll not argue about whether this is a new medium or not because that’s usually an article of faith and so the discussion is fruitless) are happening all over the place – please see the end of this letter for a small list of links about evolving how we measure this new stuff.

Secondly, I’m having a bit of a semantic problem, if niche markets are unique, or at least, not mass then how can we possibly have a standard set of metrics for them all? The metrics needed to measure one niche market would seem to me -by definition- to be different than those needed for another niche market. That said, this new iteration of communication is still new enough that we can’t see the forest for the trees. Evolving new metrics is at a point of solving specific cases and it’s still unclear how those specific cases may be extended to the space as a whole – it’ll take time to get there, if we can get there.

Third, a lot of the pieces for what you are asking for are in place, but as with many new markets they are fragemented. Technorati, Feedburner, Bloglines and other technology providers do give us (with some work) indications on authority and connectedness (I think). I’m not sure by exactly what you mean by speed but my inkling is that sites like digg and measurement tools like buzzmetrics give us a sense of how quickly any particular post or meme gets propagated throughout the ‘sphere. As this market matures, there will be consolidation and you’ll be able to go to one place (or maybe just a few) to get answers to those questions.

Finally, the page view is not dead. It is changing and may well lose it’s current weight/authority, but it won’t go away, it will be one point in a continuum of events that will each have their own validity, usefulness and authority.

You Say, “I think it’s time for a measurement summit: Bring together the measurement companies, the advertisers and their agencies (buyers), the sites’ reps (sellers), the media sites, and technology companies and let’s hammer out some standards and methods for measurement.”

Seriously, haven’t you heard of Emetrics? Jim Sterne could probably tell us for certain, but most, if not all, the groups you mentioned above are typically represented at an Emetrics Summit – you should come too.

As for standards, well the IAB works on its standards (the standards for advertising on the internet) all the time and the WAA (Web Analytics Association) is working on measurement standards too. However, there is no ‘hammering out’ with these things, unless you mean ‘banging our collective heads against a large brick wall for a very long time’. I’ve participated in standards committees at the IAB and it takes a bloody long time to get the job done. Every group wants to protect their respective interests. Everyone understands things slightly differently. Nobody is willing to agree on anything. You can’t do standards in an afternoon or a day and probably not even in a year. Developing measurement standards is a hard, frustrating, time-consuming, and thankless task (regardless of its downstream value to the market) – so be prepared.

To get back on point, why am I in such a tizzy you ask? Well, posts like this indicate (to me) that, for whatever reason, most of the blogosphere is ignoring a valuable resource – the web analysts of the world – in this discussion. Jeremiah, (in my admittedly limited view) is the only person I see actively engaging the people who have the best chance of figuring this out. Heck, you’ve probably got (and know) web analysts from the university, from the newspaper, from federated media, are you working with them on this?

Short List of “Web 2.0″ Discussion Links

Eric Peterson’s Series on Engagement (starting with #4 make sure to read them all)

Jeremiah’s Social Media Measurement Category

Sign up for the Web 2.0 Measurement Working Group

The Web Analytics Yahoo! Group

Ian S. Houston on Evolving the Web Analytics Data Model

There’s plenty more discussions out there, but these are the ones at the top of my list today.

I entreat you, and anyone else in this brave new world, to involve a web analyst in your discussion. They will help.

This entry was posted on Thursday, February 8th, 2007 at 5:33 pm and is filed under web 2.0, Web Analytics. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.