Big Data analyses have the potential to greatly improve the learning function and positively affect business outcomes—if used appropriately.
As our lives have moved online, we are all increasingly aware of the emerging, exploding interest in what has come to be known as Big Data. Big Data refers to the massive data warehouses filled with servers that store the so-called "digital breadcrumbs" of our online shopping transactions, movie selections, Facebook likes, tweets, website viewing activities, and more. Descriptive, inferential, and predictive analyses are applied to those massive data sets to look for patterns that provide insight into our desires, preferences, and behaviors. For example, retailers analyze detailed transactions to better understand customers' shopping patterns and to forecast demand, optimize merchandising, and increase the lift of their promotions. Internet companies study website information to enhance site visitors' experiences and provide advertisers with more granular targeted advertising.
Big Data analyses have ramped up everyone's expectations of what may be possible for improving accountability, transparency, and quality in all facets of our lives, from the marketplace to the workplace, to civic engagement, and at every point in the education and training value chain, including learning and development. They also have raised serious questions about the degree to which we understand which variables are truly predictive of desired action states, for whom and under what conditions they predict, and the confidence with which the predictions are made.
Put data to work
Learning and development organizations collect vast amounts of digital information in their learning management systems and enterprise resource planning systems. However, few organizations are systematic about using enterprise data at any level, Kirkpatrick or otherwise, to proactively anticipate emerging problem areas and recognize new opportunities for strategic alignment, performance support, and enterprise growth.
Today's explosive interest in analytics is fueled by the recognition that enterprise success in the era of the "new normal" depends on forward-thinking strategic alignment between goals and operations, agility, and the recognition of the proverbial right place to be and right time to be there.
According to the 2010 Gartner Re-search Report, Hype Cycle for Pattern-Based Strategy, business leaders want to move from reacting to events that will have major effects on strategy and operations to proactively seeking patterns that might indicate an impending event. Learning and development organizations simply cannot live outside today's enterprise focus on the measurable, tangible results now driving IT, operations, finance, and other mission-critical applications. More to the point, learning and development organizations have emerging opportunities for putting their data to work in new and highly productive ways that will lead to demonstrable impact and alignment with business goals and enterprise strategic directions.
Technological developments certainly have served as catalysts for the move toward Big Data. Virtual data warehouses and the cloud make it possible to collect, manage, and maintain massive numbers of records. In addition, sophisticated technology platforms provide the computing power necessary for grinding through calculations and turning the mass of numbers into meaningful patterns.
It is clear that Big Data and other more localized data analysis à la Google Analytics have the potential to inform learning professionals to anticipate the support that will be required to enable under a variety of changing market conditions. These data will be able to inform enterprises about where investments provide the greatest return. The promise of providing personalized, optimized experiences for learning and performance support that may eventually correlate with any number of taxonomic frameworks for learning success is compelling.
The one caveat is that, for analytics of any kind to be successful, there must be a framework for identifying the high-value variables that will enable the development of models that make predictions that matter in practice, at levels of confidence that make it worth the disruption that we know this innovation is going to cause.
With forethought and planning it is possible for learning professionals to mine enterprise data to better understand learning and motivational patterns, and to support the enterprise decision making that uses enterprise data in meaningful ways. Learning analytics offer opportunities to reconsider how to optimize educational experiences that promote and enable both learner and enterprise success.
The degree to which learning analytics will offer high value in your enterprise may be the degree to which learning professionals keep the following eight rules of thumb in mind.
Analytics are here today, and they are here to stay
There is no question that information can be gleaned from the transactions and interactions we leave behind along the paths of our online lives. This information can then be summarized in reports and displays that provide intelligence for making better informed decisions to shift patterns of behaviors in desirable ways. Analytics are already being used in a variety of ways in corporate enterprises such as operations, finances, sales and sales support, and perhaps most visible of all—interactive online marketing. Business intelligence techniques are being used with greater frequency across all lines of business.
Interest in learning analytics continues to explode as more tools emerge, more techniques are validated, and our collective understanding of analytics utility continues to grow. Even so, learning enterprises are still on the early side of the analytics adoption curve, especially when compared with other U.S. economic market segments such as retail, telecommunications, financial services, and manufacturing. This has much to do with the variability of definitions for what counts as learning and performance outcomes and competency assessments between and across enterprises and institutions.
Understand the big picture
Big Data is part of the bigger picture of analytics for business intelligence, of which learning analytics is a subset. There are many sources of data in your learning organization. Consider looking toward your LMSs, LCMSs, and ERPs, as well as toward local data sources (for example, course evaluation and instructor evaluation data, and content downloads) and social media metrics.
The analyses applied to data sets of all sizes will vary, but generally will fall into one of three categories: descriptive analyses, inferential analyses, and exploratory (predictive) analyses. Noted analytics expert Phil Ice has estimated that approximately 90 percent of all analytical solutions available today offer solutions based on descriptive statistics (for example, frequency distributions), with 9 percent offering inferentially derived solutions (for example, regression analysis) and only 1 percent of all solutions for learning offering true predictive capabilities (for example, Chi Squared Automatic Interaction Detection, an exploratory data analysis method that studies the relationship between a dependent measure and a large series of predictor variables that may interact).
Learning professionals would be well-served to consider the kinds of decisions that data analyses will be likely to serve, and not put their faith in the misguided belief that Big Data will directly serve up ready-to-go solutions from the mist.
Know how to crunch the numbers
Someone on your team needs to have a deep understanding of databases, statistics, and research techniques. The success of data-driven decision making campaigns depend on finding individuals with specialized skills in database management and statistical analysis. They also must possess a keen understanding of the technological requirements needed to support the extraction of just-the-right data from LMSs, LCMSs, and ERPs, as well as local data sources and social media metrics, to enable the creation of valid reliable reports and performance dashboards.
Individuals possessing the skills to turn millions of records into meaningful patterns that inform decision making with confidence are known as data scientists. Those of us who have managed big data projects reverently refer to these people as data whisperers. You will be well served to find such an individual.
Analytics are a means to the end, not the end in and of themselves
The point of analytics is to enable better decision making. People still need to make the decisions.
A useful analogy may be to consider how the Big Data processing power behind weather forecasting is used to answer specific questions. The mathematical modeling underlying weather forecasting requires Big Data support at a massive global scale, in the hope of anticipating significant global weather events from the 50,000-foot level and beyond, all the way down to the one-foot level, in search of anything that might likely disrupt commerce or national security.
And even with all that processing power in play, from the user perspective he simply wants to know such information as whether he should take an umbrella, will the plane be delayed, and will he need a fleece.
It's what we do with analytical findings that matter
Data alone do not improve enterprise success. Learning analytics do offer great promise for transforming the accountability, personalization, and relevance of enterprise learning and performance support. However, that promise will not be fully realized until we put the power of better-informed decision making based on analytical frameworks and findings into the hands of frontline learning stakeholders, including learners themselves.
Mine the data accurately
Data-based decision making presumes that one will recognize the important findings from among the huge array of analytical results. In these early days of excitement about the possibilities that analytics offer, there can be occasional yet fundamental disconnects between future-based projections of what analyses of activities streams and transactional databases eventually will offer to learning stakeholders, and the remarkable number of steps that will need to occur from the big idea to anything that had any relevancy to current marketplace needs.
The notion that all learning should be tracked, or that there is value to tracking activity-stream-level information, or that Hadoop-like pattern-seeking technologies will serve the needs of the learning enterprise, begs the question of what that capability is really worth. This is the important business value question, the "burden of knowing" that gets swept aside when the rumble of excitement about future research possibilities is the loudest sound at the enterprise learning analytics value proposition table.
There is every reason to be excited about future-looking initiatives, such as ADL's Experience API (commonly known as the Tin Can API), and the current systemic undertakings such as the U.S. Department of Energy's National Training and Education Resource. There also is value in acknowledging that there is much work to be done between the 50,000-foot view of a big idea and the one-foot view of its implementation.
For example, the work by John Campbell and his colleagues at Purdue University has resulted in a straightforward, highly effective dashboard application known as Signals, which provides college instructors with a "green-for-good," "yellow-for-head's-up," and "red-for-danger" dashboard to track student course progress. Signals's ability to inform instructors when students are drifting toward known signs of trouble has demonstrably improved student success at Purdue and the other institutions that have adopted the commercial Course Signal product now available from Ellucian.
It is useful that learning professionals acknowledge the reality of the current state of learning analytics research and application. It is hard to determine, in an empirical, methodologically defensible way, which of the various assets, keystrokes, and interactions in the massive data stream out in the ether ultimately really matter in learning.
The fact that we eventually will be able to collect, track, link, visualize, tokenize, correlate, factor, or parse data pulled through application program interfaces doesn't mean that we are tracking the right information when applying various analysis technique among the research protocols. Nor does it mean that we will recognize it as right information. Or even if we do recognize it, that we'll be able to do anything meaningful with the findings. Learning professionals are going to need to step up our collective game when it comes to learning analytics proficiency.
Be prepared to live under the 'sword of data'
When Douglas Bowman, the lead designer at Google, discovered that his design decisions were being overruled by engineers fueled by customer-preference statistics, he resigned in protest against working under what he called the "sword of data." He declared that he had "grown tired of debating such minuscule design decisions" after a team at Google couldn't choose between two shades of blue. They decided to test the two colors against 41 shades to see which color performed better.
"I had a recent debate over whether a border should be 3, 4 or 5 pixels wide, and was asked to prove my case," Bowman wrote on his website, explaining his resignation. "I can't operate in an environment like that." I won't miss a design philosophy that lives or dies strictly by the sword of data."
It's important to be mindful that there's no such thing as "sort of" transparent. Once one can see the results of the analyses being run against any number of enterprise data sources, it will be difficult to ignore what the numbers are saying.
We haven't even begun to scratch the surface of the possibilities
Using data to diagnose problems is only part of the opportunity. Analytics will contribute to informed decision making to more and more parts of the learning enterprise. Several recent examples from education provide evidence that the community is starting to pay attention to the opportunities for leveraging learning analytics in the service of high-quality, scalable, and personalized learning experiences.
The perceived power of data-driven decision making continues to alarm, provoke, seduce, and intrigue. Whatever the visceral reaction, it is clear that learning analytics will help to optimize online learning experiences that are more personal, more convenient, and more engaging.
Learning analytics have the potential to help learners and enterprises alike recognize danger signs before threats to learning success materialize. These possibilities provide the fuel that is feeding the fires of interest in learning analytics. Although effective practices capitalizing on learning analytics are still in their early stages, the number of stakeholder groups interested in discovering more about how learning analytics can support effective decision making in today's enterprises continues to keep these flames of interest burning brightly.