Analyze This!
By Ryan Sparks

New web analytics software enables e-learning developers to see what is working...and what's not.

New versions of web analytics software enable e-learning developers to go beyond usability studies and collect metrics on actual learner engagement in live courses. These new options provide e-learning course developers the means to make educated design decisions based on quantitative data derived from student behavior rather than subjective viewpoints.

You may not realize it, but most large websites track every page visitors view, how long they stay, and when and where they make the decision to purchase a product—or bail from the site entirely. The path you may follow at a particular site, along with the paths of thousands of other visitors, is captured and analyzed through web analytics software. This data collected by web analytics software helps site developers to appropriately redesign graphics and screens that improve functionality and ultimately increase usage and sales.

Web analytics software, which was once only obtainable to large e-commerce and corporate websites, is now available for free through online services such as Google Analytics (www.google.com/analytics) and Crazy Egg (http://crazyegg.com), or for a very low fee though companies like Mint (http://www.haveamint.com). These affordable resources provide e-learning developers with new options for measuring learner usage and engagement. Using these services, e-learning developers can go beyond usability studies and collect metrics that have never been detectable before. For example, developers can now uncover how long learners spend on each page of a course or calculate the popularity of a particular element of a course, such as a video, MP3, or PDF downloads.

Tracking "conversions"

Although the concept counting and ranking sections of a course by popularity is exciting, the real value of web analytics software is measuring conversions. A conversion is a web analytics term that refers to the completion of a specific set of tasks. For example, suppose a course includes a logical build that explains a process over five screens. A successful conversion would be recorded for every learner that clicked on the first screen, or Start Page, and continued to the last screen, or the End Page.

Most web analytics services provide the ability to track multiple conversions, or goals, enabling course developers to finally answer questions that could never be answered before:

• Are learners really attempting the quizzes and activities or are they immediately clicking options to advance to the next section of the course?
• Are rich media flash-based courses more interesting to learners than plain text content?
• Are learners going through each path of the course or are most learners dropping out after a couple of screens?

What to track

Getting started with a web analytics service begins with e-learning developers pasting a specific snippet of code throughout a course. This snippet of code can be pasted into HTML pages after a course has been generated into HTML, although some e-learning authoring tools provide the option to paste code into individual content pages.

Most web analytics experts recommend getting started by focusing on data points that both relate to the most important goals of the organization and that provide clear actions for change. In regards to e-learning, developers may want to start by tracking content related to the most critical learning objectives. Specifically, developers can identify the most important content their learners need to retain and then track the pages and interactions where the learners learn and apply that knowledge. Google Analytics, for example, provides the means to track these paths of activity by storing URLs and logical names for each page in a goal.

Once the goal is established, developers can monitor where learners are bailing from a path through a funnel visualization report. A version of this report is available in the Reports section of Google Analytics. This data is displayed in funnel reports helps course developers see the number and percentage of learners who continued through each step and how many learners abandoned each step.

The diagram below displays an example of a funnel report for the first two pages of an established goal. In this example, each of the green funnels represents an individual page, with the name of the page that it represents displayed at the top of the funnel. The figure to the left of the funnel reports the number of learners that entered the funnel. The number on the right shows how many learners failed to progress to the next step in the goal. Displayed within each individual funnel is the number and percentage of learners who entered the page and progressed to the next page in the goal.

E-learning developers can also drill down to the page-level to find areas where learners are skipping over important content. For example, if there is an activity on a particular page that should take a learner more than a minute to complete, and the average time spent on the page is 15 seconds, learners are likely not fully engaged in the activity.

The table below is an excerpt from the Content Performance section of Google Analytics that displays some of the most common metrics captured by web analytics software.

• Pageviews: The total number of pages viewed. Repeated views of a single page are counted.
• Unique Pageviews: The number of visits during which one or more of these pages was viewed.
• Time on Page: The average amount of time learners spent viewing this page.
• Bounce Rate: The percentage of single page visits resulting from learners that landed on this page.
• Percent (%) Exit: The percentage of course exits that resulted from this page.

Interpreting and acting on findings

Web analytics experts recommend using the data extracted from an analytics campaign to establish a baseline and set goals for the new design. This approach allows course developers to continually compare the current performance against respective targets. For example, course developers could respond to poor engagement results by swapping out learning activities or reordering the display of content. The developers could then analyze metrics such as time spent on pages and completions of goals to assess the effectiveness of their redesign efforts. The table below provides an example of how these figures may be displayed.