Performance Analysis And Web 2.0

By Allison Rossett and Joe Williams

Performance analysis involves reaching out for several perspectives on a problem or opportunity. Web 2.0 sources offer insight into what’s on the minds of the workforce and the public.

Most of the people who write about analysis describe it as an activity with a beginning, middle, and an end. You do analysis or you did it or you didn’t do it—almost always in response to a request from a customer, client, or sponsor. They ask for something, say customer service classes or help in rolling out a new product to the global sales force. The training or learning professional then swings into action to figure out what to do. That figuring out process is called performance analysis.

First Things Fast defines performance analysis as “partnering with clients and customers to help them define and achieve their goals. Performance analysis involves reaching out for several perspectives on a problem or opportunity, determining any and all drivers towards or barriers to successful performance, and proposing a solution system based on what is learned, not on what is typically done.” Once the request is delivered, analysts engage stakeholders to assure that they participate in defining custom solution for their workers, work and workplace.

Historically, the analyst examines documents and work products, interviews experts and incumbents, runs focus groups, and collects opinions through surveys and polls. Eventually, if all goes well, a targeted solution is crafted.

Web 2.0 perspectives and technologies have the potential to advance performance analysis by making stakeholders’ participation and conversation grist for analysis. If you want to know what questions sales people have about the new product, establish a wiki. If you want to know the experience the public is having with your customer facing representatives, mine their blogs.

Harold Jarche defined Web 2.0 in Learning Circuits, April 2008, “Web 2.0 is the growing set of tools and processes that allows anyone to easily create digital content and collaborate with others without any special programming skills.”  It is now possible to go online and find people exhorting, informing, complaining, honoring, questioning, and musing. As engineers, parents, service reps and sales guys and gals emote and comment, they create a rich repository for analytic inquiry.

How do we do that? How do we look at these new resources? Web 2.0 has not changed that lens for planning. During analysis, we seek two broad kinds of information: directions and drivers. When we seek directions, we are looking for the performance and perspectives that the organization and its leaders are eager to put in place. We derive directions from perspectives on exemplary or optimal performance and from current performance. Where better to find the focus for a customer service program than from complaints and kudos by customers about their reps?

We also seek information about what drives performance.  Performance drivers are the factors that block or enable performance or those that might do so in the future, when the rep is attempting to serve or the sales person to sell. Typical drivers are their skills, motivation, incentives, tools, and work processes. Consider it. You can see that all must come together to contribute to performance. Let’s apply the concept of drivers to the topic of performance analysis. Even if you possess skills and knowledge about analysis, will you do it? You’re far more likely to if you think it’s important; are encouraged by your manager or client; prodded by a criterion within your performance appraisal; and are provided some technology support in a pinch. The “knowing about it” part is necessary, certainly, but not the whole story. During analysis, you seek that whole story in order to construct a tailored solution system.

How would it work?

With all due respect for online survey tools (Survey Monkey and Zoomerang, for example) and web conferencing technologies (Adobe Connect, Dimdim, and Artifact, for example) that enable e-focus groups, e-surveying, and e-interviews, our attention here is on data that is already waiting for us online.

Web 2.0 sources offer insight into what’s on the minds of the workforce and the public. When something is working well, often it will be shared. When a concern is emerging, it most certainly will be voiced. A colleague who manages learning in the automotive industry said that he reads external customer blogs to anticipate the needs of his workforce. If customers are bemoaning pressured sales when they visit dealers or that their tires require early replacement, there are implications for learning and support services provided to the workforce.

Another colleague described her experience with Twitter. It has allowed her to find like minded colleagues within her global telecommunications company and to maintain frequent, precise communications with them. Her pool of sources has expanded, and her relationship with them in strengthened. She reported that it never would have happened without Twitter.

Mining blogs

Blogs have become a dominant force in web-based collaboration. Anyone, from the CEO of a Fortune 500 company to a dog breeder to a group of sales representatives in Latin America, can have one—testimony to their simplicity and appeal.

Using a one-to-many communication model, the blog owner starts the discussion on a topic, say the nature of customer complaints about service. Anyone aware that the blog exists can respond with comments, or comment on others’ comments. As you can imagine, participation here would be useful for moving from training to performance for customer service at this time, in this organization.

Analysts can research their topics in a more 2.0 way than going to the library. You can now search for relevant blogs the same way you search for the best price on a Mini Cooper Clubman automobile. Google, Yahoo, and others offer search engines tailored to finding blogs based on your interests. For instance, a quick trip to blogsearch.google.com revealed five blogs specifically related to customer service.

Keep up with the latest posts or refine your search by key word, or date. When you find a blog you’d like to tune into regularly, RSS (Real Simple Subscription) lets you stay current using an RSS reader. Subscribing is easy. Select the subscribe option from your browser or email client. With just a few clicks, we were reading the same content in Outlook—and will get fresh content everyday, delivered along with our email.

One of the authors was working on a study for a global accounting firm. After a teleconference focus group that did not allow sufficient time to discuss all relevant matters, we decided to move the conversation online. Time zones were also a factor here. The shared phone calls were a trial for those who had to rise at 4 AM to participate. The blog worked like a charm. We posed a few questions, members responded, and then others responded to them. It was interactive, substantive and archived. The software used was WordPress, but it could have been Blogger, TypePad, or LiveJournal.

Mining discussion groups

Discussion-group software anticipates the challenges of knowledge management through discussion groups and automated distribution lists. In a sense they create a virtual post office that distributes e-mail to the addresses of participants in a defined group. The group might be engineering managers, affirmative action officers on university campuses, members of the customer service unit, people passionate about distance learning or Dalmatians, or customers who have purchased a particular printer. Routine administrative tasks are automated, and the software archives messages for future reference. The software typically enables a moderated group, in which messages are screened prior to distribution, or one that is unmoderated, with no prior screening.

While these forms are not particularly pretty, they are focused and democratic. The opportunity comes from presenting structured questions and dialogues. What skills will people need to adjust to global teaming? What issues and opportunities are emerging for lawyers who are struggling to negotiate contracts in the developing world? What are the implications of the mobile workforce for supervisors? What are the current barriers to integration of technology into the sales process?

Post questions like these within the discussion groups. Do they respond? If not, you have probably selected a topic that concerns you, not them. If they do engage, you possess a rich and growing source of information.

Mining social networks and professional alliances

Social networks of every sort offer the performance analyst an opportunity to advance current work and long-term professional development. When you need to know more, this is a way to reach out for it.

One of the first online communities devoted to our field was the Discussion Group for Training and Development (TRDEV-L). Approximately 6,000 subscribers from 60 different countries exchange information on training and development. In 2000, the group renamed itself TRDEV and today is active in discussing matters related to the field. Real problems and people populate the listserv. They ask many questions: “How do I prepare managers to receive their first dose of 360-degree feedback?” “How do I bring automation to sales people?” “How do I learn more about electronic performance support?” “Has anybody read anything good on the subject of performance analysis?” Many fruitful responses result from these queries. One caution, however. This “group” is open to anyone who wants to join. You can imagine that someone who has written a book about performance analysis or 360 degree feedback might have positive things to say about their product.

ASTD provides a starting point for this type of discussion. When one of us typed “performance analysis” into ASTD’s discussion board search box, more than 20 related threads popped up.

Some want even more beauty and functionality. Web-based networking sites, like LinkedIn, Plaxo, or Google Groups, offer those with similar interests the ability to create online communal spaces where they can contribute ideas and stay connected beyond the boundaries of a particular project or initiative. If you are working on an initiative for your organization, part of the quest for directions can come from a review of related literature and best practices through your network. eLearningguild.com is useful if, for example, you have been asked to consider a mobile learning and support initiative for the global sales force.

The Masie Consortium’s LearningTown is a new social site for learning professionals. When we looked in May 2008, the mobile learning group had attracted 217 registrants, the rapid e-learning had 210 registrants, and the evaluation group attracted only 151 participants. Instructional design attracted by far the largest number, with 337 on board.  What are they talking about? One gent asked about software for performance support and advised vendors to stay out of the conversation. Cost is also a concern to participants. Members of the communities inquired about how long this or that typically takes and how to wring costs out of the effort.

Mining wikis

Once a curiosity looking for application, wikis have taken their place among collaborative technologies in a significant way. According to Wikipedia, “A wiki is a collection of web pages designed to enable anyone who accesses it to contribute or modify content, using a simplified markup language” Generally wiki pages are organized around a topic and provide a low-cost way to solicit and manage content from sources who need not be particularly technical. What’s interesting about wikis is that the content that emerges is democratically birthed; anybody can contribute.

The major drawback to using wikis is that same democracy. What is the motivation and expertise of the contributor? Hostile or erroneous content threaten the usefulness of wikis. Does that mean we should avoid what we find in wikis? Probably not. Nature (December, 2005) did a comparative study of Wikipedia and the Encyclopedia Britannica in which they found that the two were similar in terms of accuracy. Not surprisingly, Britannica disagreed.

Wikis can be a fertile tool during analysis. For example, a learning executive in financial services solved a problem using a wiki. The problem was the not infrequent struggle to gain access to top notch experts in the company. They did not want to allocate time to the learning group. One approach to this problems was to establish e a wiki for a small, select group of industry experts, and to have them work together to define opportunities and trends. The experts did it because they liked being in that very special group and enjoyed seeing their messages take new and easily updated forms. Their ideas were important for learning and knowledge decisions going forward.

All in one place

All this collaboration produces assets. How do you find them? How do you manage version control? You can share files, media, and calendars using sites like MS SharePoint, sites.google.com, Stixy, and www.drop.io. These sites support many kinds of collaboration, from the most basic file sharing to full-fledged virtual meeting spaces.

The IT department for a national supermarket put together an ad-hoc team to conduct an analysis to determine training needs for a new point-of-sale system. Within minutes, they created a SharePoint workspace to host their project documentation. During the life of the analysis, they easily pulled together discussion groups, created surveys, updated team contact information, and posted project announcements. At the end, all relevant project information resided in one place—where stakeholders could review and reflect on it. They managed their conversations and the resulting knowledge in a way that would be useful not only on this project, but on future projects with similar characteristics.  

A few grains of salt

In some cases, the analyst will be hanging out in chat rooms, blogs and wikis without declaring him or herself to the group. Is it right to lurk?  Robert Berkman wrote this in a blog for the USC Annenberg School of Journalism: “In 1999, journalist and author Jennifer Egan was developing a story for New York Times Magazine about gay and lesbian teenagers who were using the Internet to meet and interact with other homosexual teens. As part of her research, Egan spent several weeks visiting Web sites geared to the gay community and lurking in several chat rooms that were restricted to gay and lesbian teens.

Though Egan was in her 30s, she easily gained access to the chat rooms by entering a fake identity and age on the site's registration form. After a period of observing communication between chat room participants, Egan posted a message identifying herself as a reporter.”

Egan’s research and Berkman’s commentary are instructive. She lurked in her initial research, but soon declared herself. She wrote an article that, because of her experience, revealed the weaknesses inherent in the online registration system. Any predator could register, if wiling to lie about age and purpose. Not only are participants threatened, so is the research, as data assumed to come from teens might actually come from a forty year old pretender.

What do participants in your Web 2.0 sites presume? That is key here. If the web site is a public site for conversations about cars, lost baggage on airlines, customer service, or salmonella, lurking does not seem to be at issue. But if it is a group for new employees at Kaiser Permanente, their assumptions will be different. Many will expect privacy. And even though such a site would be a choice analysis target, analysts should announce their presence. There are good reasons for being there. Share them.  

Web 2.0 + Web 1.0

Performance analysis is guided inquiry. What we have advocated here is to look at more natural sources that exist for every reason under the sun. Look to the blogs. Consider the wikis. Chime in on the discussion boards. Examine Ning sites. Commune where the people you serve commune.

When they talk, what do they talk about? When they complain, what are the subjects? When they crow with delight, about what are they thrilled? What makes them twitter? What questions do they ask? What resources do they recommend? What topics generate many responses? Which ones lead to silence?

Web 1.0 approaches retain value. They are good, but they are not sufficient. You could say the same about Web 2.0 approaches. They too are good, but they also are not sufficient. They will reveal the passions of the crowd, or of an individual. All that is worth knowing, as long as it is leavened with data gathered in authenticated and controlled ways.