There are two categories of web analytics:

• Off-site web analytics
• On-site web analytics

Off-site web analytics refers to web measurement and analysis, regardless of whether you own or maintain a website. It includes the measurement of a website's potential audience (opportunity), share of voice (visibility), and buzz (comments) that is happening on the Internet as a whole.

On-site web analytics measure a visitor's journey once on your website. This includes its drivers and conversions (that is, when users take a desired action on the website such as request a white paper). For example, which landing pages encourage people to make a purchase? On-site web analytics typically compares data about user behavior against key performance indicators such as purchases, downloads, enrollments, or any activity that ties into the goals of the organization. The subsequent analysis of this data guides the improvements to a web site or marketing campaign's audience response.

Historically, web analytics has referred to on-site visitor measurement. However, this has blurred as vendors are producing tools that span both categories.

Web Analytics Data

Commonly measured data includes the number of times web pages are displayed (hits), how many individuals visited the website, the paths users took through a website, where abandonment of the website occurs, conversion rates (primarily for e-commerce sites), feature usage, and frequency of actions of interest, such as errors. Web analytics usually are provided through a tool that uses embedded code on the site to collect data. Most of these tools provide information about the website in charts and graphs and also provide ways to download the information for deeper analysis.

Web analytics data differs from behavioral data collected via direct observation in several ways. First, analytics provide only the outcome of behaviors (such as clicks and purchases). The long and complex chain of cognitive processes leading up to an action that can be logged can only be surfaced and understood through in-person observation by a trained researcher. For example, lack of clicks on an important link could be due to low discoverability or low perceived utility. Just because analytics show that a very small percent of users perform a certain action, it does not mean that the concept doesn't have merit. In other words, analytics cannot diagnose an issue as being a design issue versus a strategy issue. Second, although it is often tempting to view analytics data as the complete picture due to its quantitative scale, it has unique challenges in that the data is often incomplete or "unclean" due to incomplete data logging, incomplete data processing, and the pollution of the data by non-human robotic activity, which contributes outlier data that impacts averages if not cleansed.

A/B Testing

A/B testing (experimentation) can be used in combination with web metric analysis to measure the impact of design, wording, or algorithm factors. In a typical simple A/B test, two variations of an interface are presented to two randomly selected samples of users, and the impact of the two variations on key success measures, such as conversion, is statistically analyzed.

A/B testing that pits two drastically different designs against each other often raises more questions than it answers, because decision makers may not understand what aspects of the "winning" variant contributed to its success, since there are so many interaction effects. A/B testing is most powerful when the variants are informed by formative user research and in-person usability testing, and when conducted for a period of time sufficient for accounting for learning curves and novelty effects.