"Reporting Is Evil"

You might as well say Seeing is evil.

Seeing, after all, is just your eyes reporting received electromagnetic stimuli to your brain.

I can understand the sentiment behind it – I really can. It comes from a frustration with organizations and people who think web analytics is nothing more than regurgitation of data. A frustration with people and organizations that don’t "get it" and can’t "grok it". The frustration comes from web analysts within those organizations (I’ve felt it, believe me), from consultants trying to help companies get to the next level and vendors trying to sell their wares.

The problem is that reporting is NOT evil. It is vital to the web analytics process.

Analysis, web analysis, is a process – or at least it should be.

Analyses, like any process, have outcomes. The outcomes might be changes to the website, changes to advertising creative, changes in SEM strategies, changes to the web analytics implementation itself and many more that I’m not thinking of at the moment. One of the over-arching deliverables of a process is communicating its results. If the results of the web analytics process are not communicated, then no outcomes are possible. How can recommended changes to the website be made if they aren’t communicated and substantiated with an analysis of the data?

Yep, that’s my way of saying that reporting is an expected outcome of the analytic process. A process that doesn’t communicate its results is a failed process. Thus, if we excise reporting from web analytics because it is evil, we are left with a process that can give us no insight because there is no way to extract information from it. What is the point of going through the process at that point? We might as well use ‘Pin the Tail on the Donkey’ in all its blindfolded glory to make a decisions.

There are many definitions of reporting, but in this context I think that there are two that are most important:

1. Reporting is a required outcome of the analytics process
2. Reporting is a communication tool

The frustration that leads people, Eric included, into thinking and saying that reporting is evil derives from organizations treating reporting as the whole analytics process.

So, if you are spending time, money and resources on web analytics and all you are doing is reporting the data, then you probably feel like you’re not getting nearly enough value out of your investment.

Heck, without reporting there is no feedback loop in analytics and then your just running one blind test after another – throwing spaghetti at the wall as it were – until something sticks.

Reporting without analysis is just a regurgitation of facts. Analysis without reporting is impossible because then there is no mechanism for creating the feedback loop within the process.

One of the problems with the previously discussed Excel color tip is that custom colors are not particularly portable. In other words, I can easily set up the custom palette on my computer but sharing that with you is a rigamorole. Instead of the color definitions being embedded in the Excel file, I’d have to give you a template file that you could import the palette from (yech!) every time you want to look at the report with my carefully chosen and designed colors.

This led me to the question of, can I store the color definitions in VBA? The answer is, no surprise, YES! I found the following snippet in a thread over on Mr. Excel.

Private Sub Workbook_Open()
Application.ScreenUpdating = False

ActiveWorkbook.Colors(35) = RGB(0, 115, 106) ‘ Teal
ActiveWorkbook.Colors(36) = RGB(255, 255, 153) ‘ Yellow
ActiveWorkbook.Colors(37) = RGB(52, 99, 175) ‘ Light Blue
ActiveWorkbook.Colors(38) = RGB(244, 154, 193) ‘ Pink
ActiveWorkbook.Colors(40) = RGB(255, 204, 153) ‘ Tan

Application.ScreenUpdating = True
End Sub

Yeah! Now my custom colors will load whenever the workbook is opened. AWESOME. Uh Oh, wait, Excel refers to each of the fifty-six colors by an index number and by the way, the index number has absolutely nothing to do with the color’s position in the palette grid (available from the format menu).

Mr. Excel to the rescue again. In this thread, I found a cool little function that lists our colors 1-56 in color coordinated cells.

Cool Little Function: (oops I had the wrong function in there orginally – sometimes I’m my worst editor)

Sub colors()
For i = 1 To 56
With Cells(i, “A”)
.Interior.ColorIndex = i
.Value = i
.HorizontalAlignment = xlCenter
.Font.Color = vbWhite
.Font.Bold = True
End With
Next i
End Sub

The only missing piece is the original RGB value. So, with a little work in Photoshop, I created the following matrix:

The number in the top right corner of each color block represents the Index number that Excel assigns to that color (see what I mean about it not really being arranged in a particularly useful way? It looks like they messed up a couple times and put some of the numbers in order).

In the lower left hand (ish) corner of each block is its RGB value.

The first five rows represent Excel’s ‘standard color palette’ while the sixth and seventh rows represent the colors for chart lines and fills respectively.

Whew! Now I have a reference map for building out that first macro (I know which colors are which in the Excel palette when I change a particular index’s RGB value) and I have that Macro for making the custom color choices portable.

Are there other better solutions for making custom color palettes (in Excel) portable? Let us know how you’ve gotten around this issue

By the way, I am well aware that this tip is another rehash of information found elsewhere – but why should I have to go through another 2 or 3 hours of searching pain if I need this again and more importantly why should you?

Eric Peterson lobbed this my way a week ago and I just noticed it a few days ago (I’ve been hard at work on a response).

His basic question, if I may restate it, is “What do I think about using a ‘tachometer‘ to display KPI data?”

The short answer is, I love to do this – when it’s practical.

If you work in Excel, it is often not practical because the level of initial work and maintenence needed to operate a custom chart element like this violates the efficiency rule (remember, that in order for reports to be truly effective, they need to be easily repeatable so that they can be delivered efficiently and on time). In other words, I’m lazy and it is, generally speaking, too much work for not enough return.

That being said, I’ve spent the past couple of days approximating the dashboard that Eric refers to (via Dashboard Spy and courtesy of Subaru) in Excel, to see if it could be done – with the proper wherewithal.

Here’s a screen grab of the finished product:

Ok, so it can be done.

Here’s a list of resources you’ll need to pull it off:

• Andy Pope’s Gauge Work Around
• John Peltier’s Speedometer Work Around
• Apps Pro X/Y Chart Labeler
• Excel Camera Tool

Scared yet?

Ok, here’s my file for the above picture, you’re welcome to try it out (no commercial use, please), but I make no warranty as to the accuracy or stability of it.

Long story short, custom dashboard elements like the above are cool, sexy and can effectively communicate important information such as KPIs, but hopefully you have a better platform than Excel to create them in (this is where something like Xcelsius becomes helpful).

I was talking with some folks the other day and started discussing the importance of separating (or abstracting) the data in a report from the presentation of data in a report and someone asked for an example.

First, let me describe what I mean by abstracting the data from the presentation. Since I’m a web guy, I’ll give you an example via web publishing. Take this blog for instance, it is run on wordpress and at a grossly understated level, wordpress does for blogs what I meant in terms of report design.

All of the data for this site, this text, images and file locations, etc. are stored in a database (mysql) and then there are templates that query the database for the information that they need to present the data (information) in the way that I have designed it to be presented. In this situation the website data is abstracted from the presentation of said data. Contrast this with a more traditional (ok, antiquated) site that is hand-crafted, page by page, and all of the data is hard-coded into the presentation – which one do you think is more efficient?

Similarly, you can design a report where all of the data is hard-coded into the part/piece/sheet that users are supposed to look at or the data can be stored in a different location and called (or queried) into the presentation template.

There are a couple of big benefits to approaching report design this way.

1. Structuring data for manipulation and collation is often counter-productive to understanding. In other words, setting up the data so that I can easily run calculations and transforms on it is pretty much setting it up as that classic table of raw data that nobody but it’s mother can love and use. And since report design is all about providing understanding, it’s often very useful to keep the data in a seperate location from its presentation
2. Hard-coded data is hard to update. Since reports aren’t truly effective unless you can produce them easily and efficiently on schedule, hard-coding the data into the report presentation always makes that harder. So, if the data is called or queried into the presentation layer and we get our data from a refreshable query we have solved the deliverability issue – it’s now easy to produce the report on time.

Putting this strategy into an Excel context

This is very simple. In most cases, I’ll have a worksheet called (imaginatively enough) ‘Data’ where I store all of the data that I need for the report and then I’ll have one or more worksheets that provide the presentation of the data (e.g. the information and insight that the data provides).

The simplest way to call or reference the data in the data sheet is a simple reference to the datum’s location in the data sheet. There are at least two easy ways to do this.

1. Copy & Paste: Copy the cell in the data worksheet that has the datum I want and then using Paste Special –> Paste Link paste it into my presentation worksheet
2. Write the formula. The above task essentially writes a formula that says ‘this cell equals that cell’. So, I can write the formula in myself. So, let’s say that the datum I want is in cell A1 in the data worksheet. Back on the presentation worksheet in cell B2 I can write the following formula:

=Data!A1

Now, whatever datum is in Data A1 will always show up in Presentation B2

That’s the easy way, but also the most inflexible. I’m going to save detailed explanations and examples for other posts, but if you want to read ahead, using functions like:

• vlookup
• lookup
• hlookup
• offset
• indirect
• named ranges (not a function per se, but used in combination with functions and charts – we can start to get super fancy)

and maybe a couple of others provide you with a great deal of flexibility and you can create quite dynamic and robust report interfaces using them.

These are the basic principles that I apply to every report I ever author.

1

Design for Understanding
Reports, among many other important things are shortcuts. The viewer wants the report to ‘cut to the chase’ so that they don’t have to wade neck-deep through the data. So, I’m always looking to present data, KPIs, analyses, etc. in the most easily digestible way.

2

Design for Impact
Designing for understanding is just a means to an end. Our end goal is to get the report users to take action (the brass ring being the correct action). When designing a report, I am constantly looking for ways to make it impactful – ways to get the users to sit up and take notice of the information and insight it is delivering.

3

Design for Efficiency
The most beautifully crafted and fireworks-generating report will ultimately be doomed to failure if you can’t reproduce it quickly and easily. Because our business is iterative (design, test, measure…repeat) it is likely that the majority of the reports and analyses you produce must be updated on a regular basis. Therefore, if you can’t quickly update the data in the report and get it out, you’ll miss deadlines. If the report is a particularly good one and effectively drives change, then it’s actually hurting your business not to get it done on time.

By the way, these 3 rules tend to be interdependent – meaning that making gains on one often leads to losses in one or both of the others. So, it’s a fine balancing act to adequately meet all 3 rules. However, when it happens, you end up with an effective report design.

It might go without saying, but I’ll say it anyway, when you find something that works – a layout, a component, a color, etc – abuse it. Most of us have more work than we can possibly accomplish so we need shortcuts too.

But Robbin Steif’s post got me thinking…’what’s the report equivalent of an E-Mail subject line?’

There’s not a clear analog (digilog?) but we can start with the report title. If the goal of a report is to get users to take action, we need to give them a reason to read down past the title. Because I’m a slave to things that come in threes (anyone who was at Emetrics and happens to remember my presentation knows that I boiled report design down to three over-arching rules) I’ve got three rules for report titles.

Clarity: First and foremost, our report titles need to be clear. That is, they need to effectively communicate the contents of the report to the user. If we were talking about Persuasion Marketing, we might say that we are providing that initial ‘scent’ to users. Report titles that are obscure, misleading or rely on the user understanding our specialized language are going to generate that glazed-over look in the eye that we all dread.

Brevity: A variation of KISS – Keep It Short S—– A report title like “Conversion Key Performance Indicators for acisite.com from dd/mm/yy to dd/mm/yy” is very likely to make our users yawn before they’ve even started. Use as few words as possible, and with our first rule try to use common words wherever possible – you don’t really want to have to explain what “Acquisition Index” means scores of time (yes, I made that mistake once) do you?

Alacrity: (Did I get you all on that one?) Because we want our users to take action based on the information we give them in the reports, the title should have at least some feeling of a call to action. Nothing so crass as ‘Click Here’ but we want to engage our users from the very start.

Putting the three rules above to work on the example title (bad) above, something like:

• It’s clear – we know what site it’s about and we know that it is reporting (somehow) on the success (health) of the ACISITE.COM business.
• It’s also short and to the pont.
• I’d wager that anyone seeing a report with ‘Success Metrics’ in the title is at least going to have a mild interest in what’s ‘inside’.

Now all we have to do is make sure the report delivers on the promise made in its title…

One of the goals of visual report design is to disambiguate, or make clear, the differences between things (data). When working with charts, there are some choices that you have to make that will either help or hinder your effort to make things clear.

• Chart Type
• Multiple or Single Plots
• Color Choice
• Inclusion of Ancillary Information

My basic rule of thumb is this, does every element contribute to overall understanding proportionally? Note that I didn’t say equally, some elements (like the data itself) are more important and therefore take on a greater responsibility than others.

By way of example, we’ll be looking at some charts I created using basic traffic data from Instant Cognition from March 7, 2006 to March 15, 2006 (thanks to StatCounter for free basic stats).

OOTB (Out-of-the-Box or default) Chart from ExcelSo let’s walk through the choices…
1. Chart Type

A common mistake is picking the wrong chart type, in this case I selected a column chart to display time-series data – there are usually better options such as line or scatter plots.

2. Multiple or Single Plots

Another common mistake is to group data into multiple plots on a single chart on the assumption that they are similar. In thise case I am plotting all of my basic traffic stats on this one chart. One problem that arises here is the relative scale of the different plots; the page load scale is so much larger than the others that it makes it harder to distinguish trends for anything but page loads. Additionally, any time a chart has multiple plots, the amount of work that you have to do to make the information in the chart meaningful and understandable escalates.

3. Color Choice

Report Design pundits like Tufte often warn against using color (but will soften the warning by saying that it’s because if you need to print or photocopy the chart, the color – and the information that it provides – will be lost). Really what they mean is that using color well in charts, as in any design, is difficult and if you don’t apply it very carefully the color will at best distract from the information in the chart and at worst, change it in unanticipated ways. We can see that by default Excel makes poor color choices. Sure the columns are visually distinct from each other by way of color. But the plot background actually becomes the single most important element in the chart by the sheer volume of gray that it displays. Next in importance visually is the legend, followed by the horizontal rules and then the chart title while the plots themselves come in nearly last in terms of their visual importance – shouldn’t they be first in importance?

4. Inclusion of Ancillary Information

By default, Excel has included the Legend, the Chart Title, X-Axis values, Y-Axis values, horizontal rules and plot area as ancillary pieces of information – most with disproportionately large visual importance. The net effect of this OOTB experience is that the chart does not communicate well. The data is not visually important and it is difficult to interpret what the data might be saying. BTW, what are the values for each of the columns? We can estimate their values at least for the larger ones like page loads but we don’t know for sure.

A Better Solution…
1. Chart Type

Rather than using the column chart I have switched to a line chart which makes it easier to see the trends (relationships) between the data points.

2. Multiple or Single Plots

As described earlier, multiple plots on a single chart makes my job harder and being lazy I seperated the four plots into four different charts – giving each its own spot light so to speak. Note that I have resized the individual charts so that they take up about the same amount of space as the original chart.

3. Color Choice

First thing to go – plot background color – it was hurting way more than it helped.

Next I set the plot line to a solid navy blue which gives it some added importance and is clearly visible. Almost any color will work in this particular scenario (although you might give some thought to what a particular color indicates. Navy Blue is a nice cool color with neutral connotations as opposed to say Red which says ‘Danger, Danger Report User’) but be consistent – using a different color for each line doesn’t provide additional information and will slow down a user’s comprehension of the charts – they have to think about what each different color might mean.

Then I changed the horizontal rules to a medium gray and changed their thickness to the finest available – they are now more of a hint or suggestion – our brains will do the rest.

4. Inclusion of Ancillary Information

Chart titles are important – we need to keep those

Legend – I almost never use the legend because it’s very difficult to position and order in such a way that it relates directly to the plot(s) in the chart and since I am only using one plot per chart the legend is unnecessary – it doesn’t add value.

Vertical Scale – I scaled each chart appropriately for the data and then to maintain some visual continuity (which eases understanding) I set the vertical scale to divide the height into four segments – it’s the same on each chart which means that we don’t have to re-understand the scale for each one.

Data table – I’ve included the data table at the bottom of each chart so that we can quickly see the value of any point in the plot. Another couple of benefits to using the data table are that I could get rid of the X-Axis labels (they are the same in the table) and the table names the series for me. By the way, I’d like to apologize now to Tufte et. al. for using such small sets in my charts – I know it violates one of the cardinal rules; which is to use chart and graphs for large data sets only.

One of the things that you might notice right away in this four-chart solution is that three of them (page loads, unique visitors and new visitors) all follow a very similar trend while returning visitors have a very different trend which is something that is nearly invisible in the OOTB chart.

One final benefit to the solution above. Although I didn’t put it in, the four chart solution leaves me space for annotation on the plots. So for instance, I might have annotated March 9th on the page loads chart as the day that Eric T. Peterson posted a response on his WebAnalyticsDemystified blog about my Dashboard post. But we’re not interpreting my traffic in this post, just using it as example data for the charts.