September
24
2007
In his keynote at the inaugural SEMPhonic XChange Conference, Eric T. Peterson said he was going to be controversial, that he wanted to stimulate the discussion. That he was tired of the ‘me too’ and ‘that’s exactly what I think’ commentary. Fast-forward a bit and Eric goes ahead and lobs a controversial bomb my way…
"Reporting Is Evil"
You might as well say Seeing is evil.
Seeing, after all, is just your eyes reporting received electromagnetic stimuli to your brain.
I can understand the sentiment behind it – I really can. It comes from a frustration with organizations and people who think web analytics is nothing more than regurgitation of data. A frustration with people and organizations that don’t "get it" and can’t "grok it". The frustration comes from web analysts within those organizations (I’ve felt it, believe me), from consultants trying to help companies get to the next level and vendors trying to sell their wares.
The problem is that reporting is NOT evil. It is vital to the web analytics process.
Analysis, web analysis, is a process – or at least it should be.
Analyses, like any process, have outcomes. The outcomes might be changes to the website, changes to advertising creative, changes in SEM strategies, changes to the web analytics implementation itself and many more that I’m not thinking of at the moment. One of the over-arching deliverables of a process is communicating its results. If the results of the web analytics process are not communicated, then no outcomes are possible. How can recommended changes to the website be made if they aren’t communicated and substantiated with an analysis of the data?
Yep, that’s my way of saying that reporting is an expected outcome of the analytic process. A process that doesn’t communicate its results is a failed process. Thus, if we excise reporting from web analytics because it is evil, we are left with a process that can give us no insight because there is no way to extract information from it. What is the point of going through the process at that point? We might as well use ‘Pin the Tail on the Donkey’ in all its blindfolded glory to make a decisions.
There are many definitions of reporting, but in this context I think that there are two that are most important:
- Reporting is a required outcome of the analytics process
- Reporting is a communication tool
The frustration that leads people, Eric included, into thinking and saying that reporting is evil derives from organizations treating reporting as the whole analytics process.
So, if you are spending time, money and resources on web analytics and all you are doing is reporting the data, then you probably feel like you’re not getting nearly enough value out of your investment.
Heck, without reporting there is no feedback loop in analytics and then your just running one blind test after another – throwing spaghetti at the wall as it were – until something sticks.
Reporting without analysis is just a regurgitation of facts. Analysis without reporting is impossible because then there is no mechanism for creating the feedback loop within the process.
Clint,
Thank you for leading the charge and agreeing to have a conversation about my presentation, not just sitting back passively and listening. You are the man.
However, what I actually said was “Reporting is a necessary evil, but it is still EVIL!” I agree with you, reporting is necessary in web analytics, but far too many companies lose track of the need to translate those reports into SOME TYPE OF ANALYSIS. In my experience it is analysis that is fundamental to companies getting value out of their investment in web analytics solutions (be they 1.0, 2.0, or even 3.0!)
So I agree with you — reporting e/o analysis is regurgitation and analysis w/o reporting is impossible — since this is more or less exactly my message. But your customers CANNOT STOP WITH REPORTING, they need the whole RAMP (Resources + Analysis + Multivariate testing + Process) to be successful.
I’m delayed in blogging but your readers can download my entire presentation from SEMphonic X Change at this URL:
http://www.webanalyticsdemystified.com/sample/Web_Analytics_Demystified_-_SEMphonic_X_Change_Keynote.pdf
(FYI, you can see all the free stuff I am publishing at this URL:)
http://www.webanalyticsdemystified.com/wp
Anyway, I’m glad you heard the real message: Don’t just listen, THINK! You think I’m off my rocker, man, say something. You’re a stand-up guy for challenging me — even if you weren’t listening
E.
Dude,
first, thanks.
Second, I was listening … and watching. The impact of your message was that reporting is just plain evil. Equivocating with ‘necessary evil’ didn’t lessen the impact. If you’re point was the same as mine, it definitely got missed in the translation (mine at least, maybe the babel fish was out of whack)
Thanks for the link to the download, I was trying to wait for you to announce the link so I could provide it but must have missed it.
I am pretty sure I have some other questions to bring up but none that I could talk about intelligently (for me) without the reference material in front of me.
I think that we forget that reporting isn’t a dirty word. The journalist in MSM (TV, Newspaper, Radio or otherwise) has hopefully done a good job of gathering, analyzing and interpreting data and formatted that into a digestible article or expose is doing reporting.