A week-by-week overview of what this program covers:

Week 1: Using tests to increase ROI and performing a needs analysis for a test

§  Building a business case for testing

§  Activity:  Quickly estimating ROI for a test

§  Develop criteria for reviewing and evaluating vendor solutions by outlining specific business issues you wish to address and outcomes you expect

-       Detail specific business objectives you want to achieve

-       Which jobs and what uses/purposes you want to support with a test

-       Number and location of individuals to be tested

§  Consider operational issues

-       What test format should be used (e.g., Paper-and-Pencil, Computer Based, and Internet)

-       How will tests be administered

-       How will you interpret the results

-       How will you handle test data

-       Who will provide ongoing support and training for tests

§  Define constraints  (e.g., budgets, personnel resources, and technological requirements)

§  Activity:  Complete the Needs Assessment Analysis Plan worksheet

Week 2: Using job analysis data when selecting tests

§  Purposes

§  Methods

§  Use in test selection

§  Use in test validation studies

§  Activity:  Reviewing job analysis data for quality   

§  Identify sources of information about tests

§  Obtain test development information from vendors

§  Obtain information about the vendor and the support services provided

§  Obtain technological requirements

§  Research and gather information from independent sources (e.g., not the test publisher)

§  Activity:  Using the Mental Measurements Yearbook and Tests in Print 

Week 3:  Conducting a technical evaluation of the test

§  What makes a good test

§  Basics of test reliability

§  Basics of test validity

§  Overview of methods for conducting validation studies

§  Using validity evidence from outside studies

§  Case study:  Evaluating the validity of a test 

§  How to evaluate a test

-       Purpose, nature, and applicability of the test

-       How to interpret reliability information from test manuals and reviews

-       How to interpret validity information from test manuals and independent reviews

-       Test fairness

-       Ongoing research/refinement of the test

-       Data for test interpretation

-       Credentials and experience

-       Alternate forms

§  Activity:  Complete part one of the Evaluation Checklist for a test

Week 4:  Conducting a practical evaluation of the test and legal considerations

§  Try out the test

§  Does the format and medium meet your requirements

§  Information on administration

§  Acceptability to candidates

§  Time requirements

§  Scoring options

§  Cost-effectiveness

§  Overall practical evaluation

§  Activity:  Completing part two of the Evaluation Checklist for a test

§  EEOC

§  ADA

§  National origin discrimination

Week 5: Setting passing scores and evaluating the testing program

§  Setting passing scores

§  Sequence of administration

§  Tracking data

§  Using Kirkpatrick’s levels