Intelligent Tutoring Systems: The What and the How
By Jim Ong and Sowmya Ramachandran
The benefits of individualized instruction are the essence of ITS, which uses artificial intelligence to tailor multimedia learning. The authors summarize the development of ITS technology.
In 1984, Benjamin Bloom defined the "two-sigma problem," which states that students who receive one-on-one instruction perform two standard deviations better than students who receive traditional classroom instruction. An improvement of two standard deviations means that the average tutored student performed as well as the top 2 percent of those receiving classroom instruction.
The fast-growing Web-based training model has likely done little to improve classroom training--and may actually make the two-sigma phenomenon more pronounced. When using WBT, students typically rely on long-distance, asynchronous communication to get help and resolve their problems. Andy Sadler of Global Knowledge Network, a provider of Web-based IT training, says that one of WBT's biggest challenges is responding to the learner's plea: "I'm stuck, what do I do?" When faced with a problem, the student must wait for help from an online instructor, who, depending on the instructor-to-student ratio, may take awhile to respond.
The ITS approach
Imagine that each learner in a classroom or WBT setting has a personal training assistant who pays attention to the participant's learning needs, assesses and diagnoses problems, and provides assistance as needed. The assistant could perform many of the routine instructional interventions and alert the instructor of learning problems that are too difficult for it. By taking on basic assistance tasks, the instructor would be free to concentrate on training issues that require greater expertise.
Providing a personal training assistant for each learner is beyond the training budgets of most organizations. However, a virtual training assistant that captures the subject matter and teaching expertise of experienced trainers provides a captivating new option. The concept, known as intelligent tutoring systems (ITS) or intelligent computer-aided instruction (ICAI), has been pursued for more than three decades by researchers in education, psychology, and artificial intelligence. Today, prototype and operational ITS systems provide practice-based instruction to support corporate training, K-12 and college education, and military training. Indeed, the technology is now ready for prime time.
The goal of ITS is to provide the benefits of one-on-one instruction automatically and cost effectively. Like training simulations, ITS enables participants to practice their skills by carrying out tasks within highly interactive learning environments. However, ITS goes beyond training simulations by answering user questions and providing individualized guidance. Unlike other computer-based training technologies, ITS systems assess each learner's actions within these interactive environments and develop a model of their knowledge, skills, and expertise. Based on the learner model, ITSs tailor instructional strategies, in terms of both the content and style, and provide explanations, hints, examples, demonstrations, and practice problems as needed.
Research on prototype systems indicate that ITS-taught students generally learn faster and translate the learning into improved performance better than classroom-trained participants. At Carnegie Mellon University, for example, researchers developed an intelligent tutoring system called the LISP Tutor in the mid-1980s that taught computer programming skills to college students. In one controlled experiment, students who used the ITS scored 43 percent higher on the final exam than a control group that received traditional instruction. When given complex programming problems, the control group required 30 percent more time to solve these problems, compared to the ITS students. Another ITS, known as Sherlock, was developed in the early 1990s to train Air Force personnel on jet aircraft troubleshooting procedures. Learners taught using Sherlock performed significantly better than the control group and, after 20 hours of instruction, performed as well as technicians with four years of on-the-job experience.
Since those early implementations, ITSs have been developed for a widening variety of training applications. Here are some notable examples.
- An ITS developed by Alan M. Lesgold and colleagues at the University of Pittsburgh on behalf of a multinational semiconductor firm trains technicians to repair complex semiconductor chip-manufacturing equipment.
- Stottler Henke Associates, Incorporated (SHAI) developed an ITS for U.S. Navy officer tactical training using simulation and automated evaluation of each student's actions. Using this system, learners receive 10 times more hands-on experience than before. This ITS is used as part of classroom instruction and as a stand-alone system aboard Navy ships, where learners use it for self-study.
- Carnegie Learning developed a suite of ITS-based "cognitive tutors" in secondary-level math subjects. The systems, based on earlier research carried out by John Anderson and Ken Koedinger at Carnegie Mellon University, were tested in a Pittsburgh public high school, where students showed 50- to 100-percent improvement in problem solving and use of equations, tables, and graphs.
- Sonalysts, Incorporated developed a Radar System Controller Intelligent Training Aid to teach radar operating skills to U.S. Navy enlisted personnel. It has increased student throughput four times and increased job performance substantially.
How ITS works
Many traditional instructional methods present learners with facts and concepts followed by test questions. These methods are effective in exposing people to large amounts of information and testing their recall. However, they often instill "inert knowledge" that learners can recall but may not apply correctly when needed. By contrast, ITS systems use simulations and other highly interactive learning environments that require people to apply their knowledge and skills. These active, situated learning environments help them retain and apply knowledge and skills more effectively in operational settings.
In order to provide hints, guidance, and instructional feedback to learners, ITS systems typically rely on three types of knowledge, organized into separate software modules (as shown in Figure 1). The "expert model" represents subject matter expertise and provides the ITS with knowledge of what it's teaching. The "student model" represents what the user does and doesn't know, and what he or she does and doesn't have. This knowledge lets the ITS know who it's teaching. The "instructor model" enables the ITS to know how to teach, by encoding instructional strategies used via the tutoring system user interface.
Here's how each of these components works. An expert model is a computer representation of a domain expert's subject matter knowledge and problem-solving ability. This knowledge enables the ITS to compare the learner's actions and selections with those of an expert in order to evaluate what the user does and doesn't know.
A variety of artificial intelligence (AI) techniques are used to capture how a problem can be solved. For example, some ITS systems capture subject matter expertise in rules. That enables the tutoring system to generate problems on the fly, combine and apply rules to solve the problems, assess each learner's understanding by comparing the software's reasoning with theirs, and demonstrate the software's solutions to the participant's. Though this approach yields a powerful tutoring system, developing an expert system that provides comprehensive coverage of the subject material is difficult and expensive.
A common alternative to embedding expert rules is to supply much of the knowledge needed to support training scenarios in the scenario definition. For example, procedural task tutoring systems enable the course developer to create templates that specify an allowable sequence of correct actions. This method avoids encoding the ability to solve all possible problems in an expert system. Instead, it requires only the ability to specify how the learner should respond in a scenario. Which technique is appropriate depends on the nature of the domain and the complexity of the underlying knowledge.
The student model evaluates each learner's performance to determine his or her knowledge, perceptual abilities, and reasoning skills. Valerie Shute at the Air Force Research Laboratory presents the following simple example of a hypothetical arithmetic tutoring system. Imagine that three learners are presented with addition problems that they answer as follows:
Figure 2: ITS Student Modeling Example
| Student A |
22
+39
51 |
46
+37
73 |
| Student B |
22
+39
161 |
46
+37
183 |
| Student C |
22
+39
62 |
46
+37
85 |
Though all three participants answered incorrectly, different underlying misconceptions caused each person's errors. Student A fails to carry, Student B always carries (sometimes unnecessarily), and Student C has trouble with single-digit addition. In this example, the student supplies an answer to the problem, and the tutoring system infers the student's misconceptions from this answer. By maintaining and referring to a detailed model of each user's strengths and weaknesses, the ITS can provide highly specific, relevant instruction.
In more complex domains, the tutoring system can monitor a learner's sequence of actions to infer his or her understanding. For example, the Tactical Action Officer (TAO) ITS, developed for the U.S. Navy by SHAI, teaches tactical rules of engagement to officers who direct the sensors and weapons aboard cruisers. This system applies pattern-matching rules to detect sequences of actions that indicate whether the student does or doesn't understand. Figure 3 shows a report card produced by applying such rules to a student's simulation. The dark bullets describe actions the learner performed incorrectly, and the light bullets describe correct actions. The report card also provides the times at which the learner performed incorrect actions and a list of principles that he or she passed or failed in the simulation.
Figure 3: Example of a TAO ITS report card
The instructor model encodes instructional methods that are appropriate for the target domain and the learner. Based on its knowledge of a person's skill strengths and weaknesses, participant expertise levels, and student learning styles, the instructor model selects the most appropriate instructional intervention. For example, if a student has been assessed a beginner in a particular procedure, the instructor module might show some step-by-step demonstrations of the procedure before asking the user to perform the procedure on his or her own. It may also provide feedback, explanations, and coaching as the participant performs the simulated procedure. As a learner gains expertise, the instructor model may "decide" to present increasingly complex scenarios. It may also decide to take a back seat and let the person explore the simulation freely, intervening with explanations and coaching only upon request. Additionally, the instructor model may also choose topics, simulations, and examples that address the user's competence gaps.
Authoring tools lowering costs
Like multimedia authoring tools, ITS authoring tools can simplify the development of tutoring systems (see Figure 3). One pioneer is a powerful tool developed by Douglas Towne and Allen Munro at the University of Southern California. Their systems incorporate simulation, graphics, and intelligent tutoring capabilities to teach equipment maintenance and other procedural tasks. Other ITS authoring tools are currently being developed at academic and commercial institutions.
ITSs can complement multimedia CBT by providing opportunities for learners to solve realistically complex problems that require them to apply information presented in the CBT. Though ITSs typically use interactive simulations and knowledge-based software, some ITS concepts can be incorporated into courseware developed with CBT authoring tools. For example, CBT courses can present problems and use mouse-sensitive graphics and carefully designed multiple-choice questions to infer a participant's understanding of the content, then branch accordingly. In addition, newer authoring tools provide scripting languages and support integration with external software modules (for example, Windows Dynamic Link Libraries) that can be used to incorporate ITS software capabilities into multimedia CBT.
ITS and simulation
Though the first simulators were developed for flight training and complex device operation, simulation-based training is beginning to be applied in other areas, including such soft skills training as selling, negotiating, and working with co-workers.
In many simulations, learners can benefit from ITS tutoring. They may take far longer to learn missing knowledge and skills without coaching from a human instructor or an automated tutor. Says Mark Esdale, director of technology at WICAT, a vendor of flight training simulators, "Simulation without an instructor is almost useless." ITSs provide a way to enhance simulations with one-on-one instruction.
ITS is still perceived by many as a technology of the future, but the rapid growth of learning software and artificial intelligence is making it a viable option. ITSs for technical training is a particularly promising field, and Web-based ITS will soon add substantial improvements to that popular training medium.
Jim Ong is an engineering group manager and Sowmya Ramachandran is a project manager with Stottler Henke Associates (www.shai.com), a San Mateo, California-based consulting firm specializing in ITS and artificial intelligence.