Kimberly T. Staten is an adjunct graduate professor for the Department of Educational Technology at Boise State University. She also is a faculty assistant in the Graduate School of Management and Technology at the University of Maryland, University College for the online Cyber Security program. Staten has more than16 years of experience training and developing faculty, staff, and students in higher education, government employees, and military personnel while serving in the U.S. Air Force. After working in the training and development field for more than 15 years, she has seen the increasing demand for e-learning and the need for effective training for faculty to deliver quality instruction to learners. She is passionate about mentoring underserved young people, and her dream is to establish an organization to cultivate her passion for educating and mentoring underserved youth focusing on improving scholastic aptitude in science, technology, engineering, and mathematics (STEM).
Q. What is the advantage of using a ROI system?
Staten: In the ever-changing information age of today, training has become big business, and technology is a huge part of that paradigm shift. According to a 2005 HR Focus article, the top trends in training and development for organizations are acceptance for the need of training, emphasis on workflow learning, technology-enhanced learning, return-on-investment in training, rapid e-learning, and outsourcing.
According to the 2004 ASTD State of the Industry report, "organizations are linking learning to performance and the bottom-line, and they understand the relationship between creating a skilled workforce and achieving enterprise-wide success." These organizations are investing more money for training and employee development, thus indicating the need for training. In 2010, the American Society of Training & Development (ASTD) estimated that U.S. organizations spent $125.88 billion on employee learning and development annually, with nearly two-thirds of the total ($78.61 billion) spent on the internal learning function. This data signifies that training expenditures were directly linked to the overall profitability for several organizations, which is a major advantage of using ROI.
Q. What is the value of using a ROI system as a standard to measure the effectiveness of training faculty to deliver quality e-learning instruction?
Staten: "In its simplest form, evaluation should address the question of whether the training program achieved its objectives," Jon M. Werner, Randy L. Desimone, and David M. Harris wrote in their book, Human Resource Development. The authors also assert that Jack Phillips emphasizes several instances for using evaluation: determining if the objectives are met, identifying the strengths and weaknesses of the program, determining costs - benefit ratios - and determining the participants for future programs. Although several different evaluation models exist, there are many similarities between all of them as well as differences. However, Kirkpatrick's four levels of evaluation are credited as the most influential evaluation tool in the training and development field. Other authors who also have developed training and development evaluation models are Phillips, Robert Brinkerhoff, Roger Kaufman, and Elwood F. Holton III.
Q. How would an organization effectively implement a ROI solution?
Staten: According to the 2006 ASTD State of the Industry report, "ROI calculations were performed routinely by some organizations and, for others, ROI was used when a learning initiative represented significant investment or was tied to long-term organizational strategy." In the case study conducted at a southwestern two-year college, the research focused on evaluating training for faculty to deliver effective e-learning. Phillip's ROI model was use to evaluate the costs and effectiveness of implementing new training for faculty. The structure of the Data Collection Plan is illustrated in Table 1.
View Table 1.
The evaluation process was developed utilizing Phillips' ROI model (see Table 2). Jack J. Phillips and Patricia Pulliam Phillips (2003) outlined the following steps to determine the feasibility of the initiative and the potential ROI:
- action steps
- potential results
- a ROI Impact Analysis.
The faculty training workshop focused on online pedagogy such as lesson planning, content delivery, engaging students, assessment, and student support. The implementation process would require an initial investment to procure the necessary hardware and software for the course management system to deliver the course content.
The ROI potential savings could reach nearly $13,000 with an ROI of 68 percent. One of the benefits of this analysis is the intangible benefit for faculty job satisfaction, which reduces frustration and burnout thus, encouraging high performance. A complete explanation of the ROI is detailed in Table 2.
View Table 2.
The faculty training program will orientate all faculty to the online environment while equipping them with the basic skills necessary to deliver quality e-learning instruction. After conducting an analysis and an evaluation, the organization is in a position to effectively implement change to address the human performance issues.
The training program delivered via the course management system was evaluated based on Kirkpatrick's four levels of evaluation and Phillips' ROI evaluation model. Successful implementation of the training program will determine the organization's readiness and its ability to make the transition to an e-learning environment.
(2005). What to Do Now That Training Is Becoming a Major HR Force. HR Focus, 82(2), 5-6. Retrieved from Business Source Alumni Edition database.
American Society for Training & Development. (2004, December 1). ASTD Releases its 2004 State of the Industry Report: Use of Technology on the Rise; Spending Remains Constant [Press release]. Retrieved from /NR/rdonlyres/DD55B2EE-0823-44C8-BED5-17BCC5448581/0/ASTD2004SOIRrelease.pdf
Desimone, R., Werner, J., & Harris, D. (2002). Human resource development. 3rd Ed.
Paradise, A., Rivera R. (2006). State of the Industry: ASTD's annual review of trends in
workplace learning and performance. Alexandria, VA: ASTD.
Phillips, J., & Phillips, P. (2003). Using action plans to measure ROI. In Chevalier, R. (Ed.), Human performance technology revisited. (pp. 262-271). Silver Spring, MD: International Society for Performance Improvement.
Disclaimer: The views expressed are those of the author and do not reflect the official policy or position of the U. S. Air Force; University of Maryland, University College; or Boise State University.