Managers across state and federal agencies face outsourcing of jobs. Results-based certification courses and processes help them establish quality control.
Departments of Transportation (DOTs) across the country are challenged to maintain their roads and bridges in the face of large fiscal budgets cuts. Because the work does not diminish with the budget cuts, much of the labor and inspection processes are being transitioned to personnel in private industry.
Certification programs help ensure consistent levels of quality across individuals who inspect the manufacture, delivery, and application and maintenance the roads and bridges. The Virginia DOT took a "performance-based" approach during this tough transition period to build a certificate program with measurable results. Managers in other agencies who face similar jobs outsourcing may find the certification course development process outlined here helpful to establish quality controls.
Although roads and bridges continue to be built in Virginia, most of the funding and focus is on infrastructure maintenance. VDOT needs competent, experienced inspectors to inspect the asphalt plants producing the materials as well as field application of the product.
VDOT already had a Materials Certification School (MCS) to train and certify inspectors in all aspects of road construction and maintenance. MCS was established more than 35 years ago and has successfully certified inspectors and technicians. It used 11 different certification guides and workshops to accomplish this task.
The overall VDOT/MCS goal and mission is simple, yet important because VDOT must ensure that:
- Private industry and VDOT personnel consistently meet the VDOT quality specifications of the job requirements for all work performed on their roads and bridges.
- All personnel meet or exceed federal requirements (CFR 637) so that VDOT can secure federal financial support for its roadways and bridges.
- Safe roads exist for all people who travel on the roadways and bridges in the Commonwealth of Virginia.
Updating for Performance
Although VDOT had certification guides for each of the MCS certificate programs, the participant guides and workshop materials were written primarily for internal VDOT personnel. In addition, they were quite dated in their look and feel. The overall design and structure limited learner's ability to quickly find information, identify what was most important, capture key concepts and terms, and make the information their own. In addition, the design and content presentation was more "subject matter-based" than "performance-based," an important distinction that affects students' ability to retain and apply content to produce desired results.
VDOT needed to apply a results- or performance-based approach to the learning and certification process. Performance-based training is both outcome-centered and learner-centered. In other words, it improves the performance of individuals, which in turn adds value to both the skill set of the performer and the business results of the organization. The table above illustrates this distinction.
Platinum Performance Partners (PPP) has had a long-standing working relationship with VDOT and won a bid to assist VDOT and MCS to update the materials certification workshops. The first project was to refine and redesign the Plant Asphalt Certification workshop materials and the Field Asphalt Certification workshop materials. Project tasks included designing and developing materials for two workshops, two certification study guides, two lab proficiencies, and two facilitator guides.
Redesigned materials needed to address the two critical measureable factors that affected the overall asphalt process:
- quality of asphalt materials produced
- effective and efficient delivery and application of the product.
The PPP team used the Performance DNAâ„¢ methodology, a validated process for the design and development of performance-based curriculum. As shown in Figure 1, this process began with the business analysis phase. During this phase, we met with VDOT and MCS to align the approach for the project goals. This helped build the partnership for success throughout the project.
Understanding Goals and Roles
After project goal alignment, we conducted performance analysis to understand VDOT's goals and confirm the job roles with the most impact on these goals and the desired outcomes produced by each role. We then completed key performer analysis activities. This required interviewing individuals in each job role who were above average in their ability to deliver desired results.
The output from this analysis was the outcomes and key work processes reported by the inspector and technician roles involved in the production, delivery, and application of asphalt. The influence analysis involved examining the effect the current training materials had on performance and identifying ways to improve their impact. Finally, we consolidated our analyses and produced project reports.
Telling Ain't Training
Our team now understood the outcomes required and the work processes that had to be completed by each target job role to produce the outcomes. Before designing new materials to produce the required performance, we needed to better understand the current state of the content. We began with a process that was critical to project success—a thorough content analysis and "shred" of the existing participant guides and presentation material.
A well-known saying, "telling ain't training," described our initial reaction to the existing materials, which reflected heavy subject matter presented through blocks of heavy text and subject matter expert (SME) lecture. We now knew what we had to work with, and where improvements were needed to create materials that were performance based.
During this analysis, we identified primary knowledge chunks and key concepts that aligned with the outcomes already identified. Using a table, we associated existing figures and tables with the key concepts and noted which of these needed additional work to improve clarity and effectiveness. In addition, we noted areas of duplicate coverage and inconsistency both within and across chapters, and content that appeared to be superfluous to the outcomes specified ("nice to know" information versus "need to know"). This content analysis also enabled us to identify ways we could reorganize the content and modify headings to improve the learner's ability to use it. Figure 2 illustrates this process.
We shared the output of the content analysis with the assigned SME. This gave the team a starting place from which to work. It also allowed us to identify early on new content that was to be secured, new figures required, and figures or text that required reworking. Our next step was to solidify a design for the participant study guide that would improve the participant's ability to
- find information
- extract critical information (best practices, safety information, and procedures) and key concepts from other content
- mentally organize the new knowledge
- remember and recall information for the certification test
- transfer learning to the jobsite.
Within the guide's chapters we used a variety of techniques to improve adult learning, depending on the chapter content and type of information being conveyed. Some of these included
- step-by-step procedures and photos directing "how to" activity
- decision tables for "if-then" situations
- graphics to provide context and additional information
- step-by-step examples for completing sample problems
- worksheets to guide procedural calculations and a reference section of formulas used in calculations.
In the certification study guides, a heading clearly establishes what the content is about, steps for proper storage are detailed with adequate white space separating ideas, and a figure provides a graphic example. At the bottom of the page a graphic symbol calls out a best practice for participants.
As we completed each component of the certification study guide, our design team worked closely with the assigned SME to
- ensure accuracy and integrity of the content
- guard against "subject matter creep" (for example, the tendency for unnecessary "nice to know" content to be added back in to the learning material)
- gain SME support and buy-in for the design.
In addition to the certification study guides, the team also refined PowerPoint slides for the certification class, a facilitator guide to keep SME instructors focused and on track, and a workshop agenda.
Results and Reviews
As performance-based training materials, VDOT's new Asphalt Field Certification Study Guide, Asphalt Plant Certification Study Guide, and supporting workshop materials enable workshop attendees to
- use chapter objectives to set an expectation for learning
- better understand the structure of the content
- more easily identify important content
- understand and define new terms
- follow lab procedures more successfully
- use the materials more effectively to complete certification tests
- practice what they learned in lab proficiencies
- complete a written exam based on the content learned.
However, the learning—and the work—does not stop there. In the field, both the private-industry inspectors and VDOT inspectors are reviewed on a random basis. These reviews ensure they continue to make their inspections according to strict VDOT specifications for a particular job and meet CFR 637 requirements. They are required to take annual online proficiency tests to keep their skills honed. In addition, they must recertify every five years.
The challenges of managing VDOT, the third largest state-maintained highway system in the country, are increased by the reduced budgets and reduced staffing—problems all state DOTs face today. By taking a "performance-based" approach to the design and development of their materials certificate program, VDOT is leading the way in producing qualified VDOT and private-industry inspectors and consequently, safer roads, tunnels, and bridges.
