A Short, Personal History of the WAA Standards

While getting caught up on my blog reading, I saw that Avinash Kaushik noted that the fourth version of the standards document of the Web Analytics Association was now available. It was fun to read the document and see how it's evolved, since I was there when it was born.

Three years ago, Jason Burby and I were appointed Co-Chairs of the Standards Committee, given a blank sheet of paper, and told, "Go figure it out." One of my hot buttons was to infuse the standards with a framework, so that it wasn't just a list of definitions, but also a way of looking at the problem. That way, if a company felt they needed a metric that wasn't listed, there was a framework they could use to create their own. Jason and I wrote that there were three types of web analytics metrics: counts, ratios, and key performance indicators (KPIs). We also noted that a metric can apply to three different universes: aggregate (total site traffic), segmented (a subset of site traffic), and individual (a single site visitor).

Jason and I sent the initial document around, and then the fireworks began. I remember civil but heated discussions with folks such as Eric Peterson about the difference between counts and KPIs; weren't we over complicating things; etc., but Jason and I held our ground. With that battle won, the Standards Committee started defining terms, a slow and arduous process. I then had to step down as Co-Chair when I joined Burton Group in February 2006. (To ensure analyst impartiality, Burton Group won't let its analysts serve on standards making committees).

So it was with some trepidation that I opened the latest version of the standards document, worried that the framework contribution was no longer there. Happily, the framework wording remains, but with the addition of a metric type called "dimension." Also happily, the terms have evolved quite a bit. Here's the November 2005 definition of "Unique Visitors" and the August 2007 definition:

November 2005

Calculation: The number of inferred individual people, within a designated reporting timeframe, with activity consisting of one or more visits to a site or the delivery of pushed content. A unique visitor can include both: (1) an inferred individual that accessed a site, or (2) an inferred individual that is pushed content and or ads such as e-mail, newsletters, interstitials and pop-under ads. Each individual is counted only once in the unique visitor measure for the reporting period. 

The unique visitor measure is filtered for robotic activity prior to reporting and these measures are determined using a cookie-based method. For sites that utilize the unique cookie approach to determine visits or recipients of pushed content, this information can be used as a basis to determine unique users across a reporting period. The use of persistent cookies is generally necessary for this measure. An algorithm is used to estimate the number of unique users on the basis of the number of unique cookies. The algorithm should adjust the unique cookie number therefore accounting for multiple browser usage by individuals and multiple individuals using a single browser.

Comments: Unique Visitor differs from Unique Authenticated Visitor in that it infers a person, rather than guaranteeing the visitor is a person by requiring a logon. For example, if John Smith visited a site twice within a week, once from work and once from home, he would be counted as two Unique Visitors (two different PCs, two different cookies, perhaps two different IP addresses) even though he is one person. 

Although it is less accurate than Unique Authenticated Visitor, Unique Visitor does not require a logon and so has much wider applicability than its counterpart. Note that Unique Authenticated Visitor should be used in place of Unique Visitor when it is possible to do so — within intranets and extranets, for example.

August 2007

Definition/Calculation: The number of inferred individual people (filtered for spiders and robots), within a designated reporting timeframe, with activity consisting of one or more visits to a site. Each individual is counted only once in the unique visitor measure for the reporting period.

Comments: Authentication, either active or passive, is the most accurate way to track unique visitors. However, because most sites do not require a user login, the most predominant method of identifying unique visitors is via a persistent cookie that stores and returns a unique id value. Because different methods are used to track unique visitors, you should ask your tool provider how they calculate this metric.

A unique visitor count is always associated with a time period (most often day, week, or month), and it is a “non-additive” metric. This means that unique visitors can not be added together over time, over page views, or over groups of content, because one visitor can view multiple pages or make multiple visits in the time frame studied. Their activity will be over-represented unless they are de-duplicated.

The deletion of cookies, whether 1st party or 3rd party, will cause unique visitors to be inflated over the actual number of people visiting the site. Users that block cookies may or may not be counted as unique visitors, and this metric is handled in different ways depending on the analytics tool used. Ask your tool provider how blocked cookies are managed in their tool: it is important to understand how this impacts other metrics with regard to these visitors.

As you can see, quite a difference. So while standards can seem to have a certain inevitability when you read them on the page, that is far from true--an opposite decision here, a tuning of the language there, and they can turn out quite differently.

If you are interested in or practice web analytics, I encourage you to read these standards: they offer some solid landmarks in a perpetually shifting landscape.

Comments

[Eric T. Peterson]

Guy,

Interesting history, I had forgotten about Jason, you and I going around on counts versus ratios. I loved the document but would love your feedback on my post at http://blog.webanalyticsdemystified.com/weblog/2007/08/congratulations-to-the-waa-standards-committee.html

Glad to hear you're still thinking about web analytics!

Eric T. Peterson
http://www.webanalyticsdemystified.com