The role of the Federal Reserve Bank is three-pronged: monetary policy (think: Greenspan), financial services (think: cash and check processing), and bank regulation. To address bank regulation, the Fed has a staff of highly skilled bank examiners who monitor and regulate Fed-member banks throughout the United States.

Bank examiners come to the Fed with a wide range of education, experience, and knowledge. There are about 2,000 examiners throughout the Fed’s 12 banks, and they must all complete a curriculum of courses, which include approximately 15 classroom courses, as well as a host of prework, postwork, self-study, and on-the-job training. At the end of training, examiners complete a comprehensive proficiency exam to become commissioned examiners.

In early 2002, the Fed’s management expressed an interest in developing training for examining a bank’s internal controls to prevent or identify fraud. The target date was December 2002, which gave the team approximately six months to complete the project.

As with most new projects, there was an initial analysis and design phase. To get started, we asked SMEs to assist with content development. We brought 10 of the system’s experts on internal controls, from throughout the country, to St. Louis for a two-day, kick-off meeting. We discussed goals, materials that already existed, and what the finished course or tool might look like.

With the help of the SMEs, we identified the content, which derived from a three-day traditional class. We organized that content into modules and lessons (some modules had one or two lessons, some had six lessons). Also, from our interviews with past students, we found that the participant’s favorite part of the learning experience was hearing stories from the experienced instructors. For example, regulations and rules can come to life when examiners hear what a senior staff member does in Boston or Phoenix, Arizona. To address this, we built a mini case study element into each lesson.

Finally, we set up a conference call schedule to review progress and receive development input from SMEs.

After the analysis and information-gathering phases, it was time to start designing the tool. While we knew that there was specific information that we needed to cover, we also knew that we that we wanted to design a tool that was easily accessible by the lowest common denominator workstation—but not sacrifice instructional value. For example, many bank exams take place in small towns, some of which have one phone line in a conference room. Therefore, we quickly concluded that using a 56k connection to access online content was not an option. Although many banks have very advanced technical capabilities, we had to consider how examiners would access the tool on a majority of jobs. For instance, Fed examiners have laptops, which they use on the job, to reference materials on their desktops and to track comments and assessments.

So, we knew we wanted something online, but we also needed to keep the content of the tool in context. Think: modular. Unfortunately, that notion was in direct contradiction to the idea of offering a downloadable option. What to do? We settled on creating individual static Web pages, but we used a single Macromedia Dreamweaver template with a series of carefully structured library items.

The instructional technology group was tasked with managing all of the technical development, as well as some of the development of graphics and interaction. First, we developed a storyboard template to get these lessons into a new, yet-to-be-designed tool.

Next, our group spent a lot of time talking about the tool, what is should do, how it should work, how users would get content, and so on. We said, "It’s more than just learning, and it’s more than just reference." We started referring to it as a hybrid tool. So finally, after plenty of analysis and brainstorming, then we sat down at a PC, launched Fireworks and Dreamweaver, and started putting ideas on screen. The collaboration was great, and we soon had a tool we could send to the graphic designers who polished the layout and concept into a tool that was easy to use, visually appealing, and professional. (Of course, solid instructional design is key, but having a nice looking tool or course can make a major difference in use and adoption of learning.)

So with a shell ready, we continued to work with the 10 SMEs on 38 lessons, each of which would take a user about 15 minutes to complete. Some lessons were delivered in early, some right on time, others late. Once SMEs sent in their content, we still had a lengthy process of editor review and IT review. Then we sent lessons back to the SMEs to review (to ensure the content was still accurate and appropriate). Finally, it was returned to our group to actually build.

Finally, with the content built and online, SMEs would go out again and review their content in the tool, in context of this hybrid reference/training tool that we had all built together.

We developed a comprehensive pilot test strategy, complete with scenarios that users would consider as they launched the tool. One question was, "The examiner in charge has just identified a very troubling finding in the ledger book at Founders’ Bank and Trust, and you have been tasked to follow up on potential check fraud schemes; using this tool, what might you do?"

We had users throughout the Fed assist us with this pilot test, and we collected survey data online from each learner: 1) what scenario he or she was referring to, and 2) what lessons in the tool he or she might access. We also asked the standard "did you like it, what else would you want to see" questions.

Positive pilot data validated our design and gave us a few ideas for tweaks to complete before the official launch.

After the test, the course was launched—on time. Since the launch, survey data continues to remain positive. And plenty of learners still offer suggestions along the way, allowing us to make subtle improvements to the tool.

Around that time, we started working with another course committee on a series of regulation and policy manuals that examiners read, called BEST modules. This is a series of about 20 PDF files, some as long as 100 pages. When surveyed, the examiners said that the content was great, but that they wanted an easier way to access and reference the great information in those modules. With the success of "Examining Bank Operations" under our belts, we decided to work with the course committee to build another hybrid tool. We had learned some lessons on the past tool, and knew what changes we wanted to make.

As before, we started planning on the next tool, this one called "STaRT" for Supervision Training and Reference Tool, by analyzing the content, design, and development structure. The material was very similar to the "Examining Bank Operations" tool; there was a large amount of initial text, some references, a possibility for case studies, and self-checks.

We thought that even though the design might be similar, the development model could look different. We also had to find a different way to access and use knowledge from the SMEs. We wanted the best and brightest for each module, but we didn’t want a long development cycle. Rather than having a large team of SMEs on multiple modules, we created a small team of SMEs on one module. This reduced their commitment to the project and enabled us to work with them over a shorter period of time for a specific content area. We also rebuilt the storyboard templates to include not only the sections of the tool, but also some editing notes for each section to reduce the amount of times each person had to review each lesson. With our development strategy hammered out, all we had left to focus on was the details of the design.

Although we liked the layout of the first tool, we wanted to add in functionality for a true search. We also needed to consider how we would build all of the pages for the tool. We decided to develop a solution that used ColdFusion to assemble the pages for us, so we literally had one page that acted as a presentation page and assembled the appropriate assets as a user selected a module and a lesson. We still had to create the content pages in HTML, but we applied the Cascading Style Sheet (CSS) to the presentation page, in order to make all of the content pages simply that—content.

We still had the requirement to make the tool downloadable, though. After evaluating several utilities that could build static versions of dynamic content, and allow search capabilities without requiring a backend search index, we settled on some tools and felt our strategy was solid.

Finally, there was the potential that we would develop 20 modules over a two-year period. In the end, we used a phased approach in which every six weeks we would deliver two modules within a single week. This phase would last approximately six months, spanning the end of 2004 through 2005. During the non-delivery periods, we would develop and launch content (See table 2004 Development + Launch Cycle). By the end of 2004, we had 10 modules, totaling more than 100 lessons.

We have started work with the course committee on "Examining Bank Operations" and have decided to move all of its content into "STaRT" model. We’re also thinking about the next version of these tools. We would like to add more interactivity, such as an "Ask the Expert" function or perhaps videos of actual bank staffing action. Imagine a video on fraud with the "tells" of when people are not telling the truth and follow-up questions that senior staff have used before to get to the truth.

There’s nothing holding us back now. The shell is there, and it’s working great. More important, it’s giving people a tool that they can use on the job. But we’re not done—there’s always room for improvement.

The Two Tools Compared

Erik Soell and Jon Basden are instructional technology managers at the Center for Online Learning for the Federal Reserve Bank of St. Louis. Contact Erik at erik.g.soell@stls.frb.org; contact Jon at jonathan.c.basden@stls.frb.org.

To view the tools in action, visit Examining Bank Operations website and the STaRT website.