Getting Down to the Right Numbers, Part 3:
Churn, Churn, Churn!

Does Your Site Have too much Bounce?

Our last Analytics Brief (Itsy Bitsy Spiders), addressed how to measure the quantity of visits to a web site and explored how web traffic software under-reports the visits of automated, non-human visitors traffic, (i.e., spiders, link checkers, bots, etc). Expect to find from 20% to 40% of total traffic is from non-human visitors when properly filtered.  Our case study illustrated the point tellingly: Of 381,300 visitor sessions during a six-month period, only 265,400 were driven by real human beings, a difference of 115,900 or 30%! (See Table I below).

Determining the correct number of visitors is not a trivial matter since accurate traffic data is critical to good business decisions.  However, this is just the first step.  In this brief we will:

• Explain why all traffic is not of the same quality;
• Demonstrate the techniques used to identify quality visitors; and
• Show how to use this information in your web site evaluation efforts.

All Web sites receive superficial traffic � visitors who land on a single page (often the Home Page) and then abruptly depart. When surfing or searching the web, we tend to have short attention spans. If something doesn�t grab our attention after a quick scan of a page, we leave.  Frequently, this is the case for visitors clicking on a link provided by a search engine. Realizing the site isn�t aligned with their search goals, they quickly leave � typically within 30 seconds.  This scenario, repeated thousands of times a week, adds up to multiple web site visits that have little or no value, what we call churn.

Consider the following scenario: Two individuals walk into a clothing store. The first buys a shirt and pair of pants. The second walks through the entrance door, glances around and then immediately leaves. The two visits are entirely different in nature and of vastly different value to the store.  We would not expect the store manager to describe the person who abruptly left as a customer.

This type of scenario occurs many times a day on every web site.  Yet we typically report web statistics as if all visitors had equal value. 

Let�s be blunt!   No value has been created or exchanged for either the visitor or the site operator when someone lands on a site and then immediately leaves.  Accurate analysis of visitor traffic should reflect this distinction in quality of visitor traffic.

Many web analytics consultants recommend tracking the number of visitors who do not go beyond the Web site home page.  The metric they commonly use to do so is variously referred to as the Home Page Reject Rate or the Home Page Effectiveness rate.  While these can be useful metrics, Hillwatch prefers to consider all aspects of site churn � not just the Home Page.  It is for this reason that we use Site Bounce Rate (although we also recommend that clients track the Home Page Reject Rate.) 

The Site Bounce Rate is calculated as follows:

(A) One page visits of 30 of seconds or less/total visitor sessions X 100

Note the 30-second time factor in the Site Bounce Rate.  Research reveals that users who land on content-rich pages that meet their needs do not have to go beyond the one page to have a satisfactory user experience.  However, they have to stay for more than 30 seconds to fully absorb the information on the page.  So a 30 second threshold is a good qualifier to use.

Getting the Bounce Out of Visitor Traffic

Large, well advertised e-commerce Web sites with strong brand names report Site Bounce Rates in the 30%-40% range.  Government and public policy sites Bounce Rates can be much higher.  If your Site Bounce Rate is high it could mean the site is not optimized for search engines or your marketing is targeting the wrong visitors.  Home Page Reject Rates that are high might reflect poor home page design or usability issues. 

But even if your web site is doing a good job in all these areas, expect to experience a large number of superficial transient visitors. It is just the way the Internet works.

Table I lists a number of statistics covering a six-month period for the case study discussed in our two earlier Analytics Briefs. Analysis of traffic data reveals a Site Bounce Rate of 61% (based on number of visitor session less all non-human visits, Hillwatch filters), i.e., 61% of all visitors looked at only one page for less than 30 seconds and left.  

Table I: Comparison visitor data

So even though the web traffic reports showed 381,315 visitor sessions over that six month period, more than 70% of those visits were meaningless either because they were automated traffic or they were superficial visitors who entered and then immediately left the site.

It is Quality � Not Quantity!  A big lesson of the dot-com meltdown was to look beyond the hype about the �number of eyeballs� attracted to a Web site and focus instead on the activities of the smaller number of valued visitors. However, most managers of public sector, non-profit and many private sector web sites are still at the �counting eyeballs� stage.

Hillwatch believes this will change in response to several emerging factors:

• First, there is a growing movement in governments demanding greater accountability for online investments.  The UK, the OECD and US have all released reports critical of the lack of meaningful, accurate and actionable metrics and evaluation criteria for government web assets. 
• Secondly, as the Canadian public service enters another period of expenditure review, expect more critical questions about the cost of creating and managing government web properties and greater demand to demonstrate meaningful outcomes.
• Thirdly, senior public sector managers are becoming more web literate.  With a rise in understanding of what it is possible to measure and evaluate will come a demand for analysis and reporting that goes far beyond simply the gross number of site visitors and page views. 
• Finally, a visitor session to your web site is simply not the same as a person visiting a government office or making a call to a departmental call centre. Such a visit, by itself, provides no visible proof that a citizen has actually been served. Better metrics than gross number of site visitors and page views are required to evaluate effective online service delivery. As organizations begin to take a holistic look at the sum of their service channels, they will demand metrics that are as least as sophisticated as those currently used by call centres or service centres. 

Once the senior public sector managers start to focus on the value issues around web assets, here are some logical questions they will want to raise:

• How do we specifically prove the site supports the department or
  agency mandate?
• What should we measure to understand the effectiveness of new program initiatives?
• Does our site engage a sufficient number of key stakeholders?
• Are these key stakeholders completing transactions and/or �taking up� key site content?
• Can we apportion a dollar value to this �take-up� activity?
• Do we have the right metrics to continuously manage and improve the site?
• How do we crosswalk between online and offline measurement to compare the value of the web channel to other government service delivery channels?
• How can these web metrics be better integrated into our corporate reporting (e.g., RMAF)?

The answers to all these questions are easier to find than you may think. They are right under your nose buried in your web logs. Additional effort and some ingenuity can turn log data into useful management information - and there is a big payoff.

Day after day, visitors to your web site vote with their cursors and telling you exactly which issues concern them, what information they value, whether you communicate effectively, and if you have an impact.  This interaction with your web site constitutes a running poll on how well you are doing online.  Thousands of monthly visits yields an unparalleled wealth of real-time information far more valuable than the occasional focus group or on-site survey.

In the next series of Hillwatch Analytic briefs we will explain how to segment and filter the logs to measure, analyse and report on the quality of site visits. Our first brief in this new series will be: Why bankers are not like truckers - Segmenting stakeholders.

Previous Analytics Brief:

Getting Down to the Right Numbers. Part  2: Itsy, Bitsy Spiders

Getting Down to the Right Numbers, Part 1: Hits are Good in Hockey But Not So Good on the Web

Other related Hillwatch Publications:

Political Web Sites: Strategic Assets or Virtual Lawn Signs?

Hillwatch White Paper: Government Web Asset Performance Measurement(PDF)

How We Use Government Websites