Performance measurement is precise, and thrives when definitions are clear, baselines are stable, goals are predictable, and reporting is systematic. Yet successful performance is achieved through a dynamic, discontinuous, and often messy process.
Only 50 percent of federal managers use performance information to help them run more effective programs. This percentage has not changed in more than a decade, according to two Government and Accountability Office studies conducted in 1997 and 2008. Similar results were noted in the 2011 survey of federal performance improvement officers (PIOs) by the Partnership for Public Service and Grant Thornton LLP. Yet performance information officers only spend a fraction of their time on performance management. Why not more?
Government planning offices play an important role in institutionalizing the use of performance information within their organizations. At the same time they must balance several different external demands for performance information. Figure 1 depicts how external reporting demands and internal use do not always complement each other.
What Do You Want to Measure?
Performance measures do not always reflect what stakeholders want to measure. In many instances the process may be the problem. External stakeholders do not influence the selection of performance measures even though (or perhaps because) they have a vested interest in outcomes such as cleaner air, safer roads, or better educated children. In the absence of stakeholder input, federal agencies may find themselves tracking performance that stakeholders do not care about, and not measuring what stakeholders do care about.
Although the endorsement of agency leadership is necessary to select performance measures, it is not necessarily sufficient. This is particularly true in agencies that operate in decentralized environments where influence is widely distributed. For example, the National Park Service must incorporate the views of its stakeholders for its performance measures to be meaningful.
Additionally, influence of external parties on performance is important. Many government professionals have closer relations with their peers in industry, academia, or other organizations than with agency leaders and managers. For example, a microbiologist working in a scientific agency is more likely to prioritize her work based on feedback from her professional network than on performance planning guidance she receives from the Office of Management and Budget (OMB) or her department. To capture meaningful performance information, performance planning should incorporate the views of parties in critical positions of influence both inside and outside of the agency and who have the ability to shape outcomes.
How Planning Offices Can Help
Planning offices must be sensitive to how power and influence is distributed throughout an agency’s external operating environment. Identify which groups may have particular interest and influence over different areas of performance. Based on this understanding, planning offices should engage the appropriate participants to select performance measures. This will increase the chances of selecting measures with the most promise for success.
Consult legislators and their staffs in performance measure development. Performance measures are more likely to be relevant to internal managers if they are an integral part of the discussion on issues that elected officials care about.
Precise performance measures are wedged into an imprecise operating environment.
Ideally, government performance measures should be clearly defined and remain stable over time. These characteristics are often at odds with the imprecise environment in which programs operate. Conflicts between precision and uncertainty arise when performance measures are defined, implemented, and reported, as Figure 2 shows.
Differences in policy interpretations can make it difficult to clearly define performance measures, particularly outcome measures. What at first glance appears to be a relatively simple matter of agreeing on terms can turn out to be a protracted policy debate.
Performance measures that track educational progress in our school systems illustrate this point. Debates continue about the most appropriate metrics to use to gauge student achievement. Federal, state, and local jurisdictions each weigh in with their perspectives on who should set the standards and how. Such outcome measures can only be defined through a process of policy debate and clarification. However, this can be demoralizing for program-level managers and performance management specialists who have the most expertise but may consider such policy debates above their pay grade.
Congressional vs. Program Targets
Precision and uncertainty also can clash during plan implementation. OMB and Congress would like stable, predictable, and reliable performance information to assess progress. Program managers, on the other hand, would consider performance information more immediately valuable if they had the flexibility to change measures and targets when needed to accommodate a constantly changing program environment.
The Government Performance and Results Act Modernization Act of 2010 attempts to address the need for continuous adjustment in performance measures and targets by establishing quarterly reporting on key performance measures backed by data-driven analyses. The verdict is still out on whether this quarterly reporting will provide “on-the-ground†managers sufficient flexibility to actually adjust targets and measures to accommodate changing conditions.
National performance information presented to OMB and Congress serves a difference purpose than the real-time feedback managers need to steer their programs. National performance measures should be limited to a critical few to simplify communications with stakeholders and to allow agency heads to track high-priority concerns. These national measures, however, may have limited value for program managers trying to steer program efforts at a local level.
A Federal Emergency Management Agency (FEMA) administrator commenting on the usefulness of performance metrics in managing his operation indicated that performance measures are very useful to him in gauging what part of an immediate problem he is solving. For example, in the aftermath of Hurricane Ike, performance metrics such as the rate of establishing temporary trailers for needy families were used to gauge how effectively the housing problem was addressed. But when you attempt to apply a common metric across all kinds of disaster scenarios, the utility of the more generic metric becomes more problematic, according to a 2009 GAO study of key practices at FEMA.
Planning offices can help by
- enlisting agency leadership to guide and resolve policy debates through the complex network of influences. This is necessary to arrive at clear definitions of performance outcomes. In this role agency leaders are not acting in a command-and-control capacity; rather they are serving as stewards who shepherd the policy process and facilitate a negotiated solution.
- designing and managing the performance measure definition processes. This provides a forum for participants to agree on the meaning of each term in the performance measure from a policy perspective.
- helping individual programs map out their strategies and identifying critical measurement points. This encourages the use of specific methodologies, such as logic models, to help understand how program components inter-relate to produce valuable outputs and outcomes.
- designing linkage mechanisms that can translate local performance metrics into nationally reported performance measures through meaningful methodologies, such as indexing or integration.
- Indexing: Develop a national index-based measure for reporting purposes that weights lower level supporting measures.
- Integration: Identify a national reporting measure that integrates, rather than aggregates, lower level measures.
For example, the Bureau of Indian Affairs has established a national performance goal to reduce crime in four high-crime reservations. Lower-level measures may address the supporting goals needed to reduce crime, such as recruitment, collaborations with community service organizations, or school-based initiatives.
Input From Those Close to the Action
Performance measures and targets are often established by national program managers without sufficient input from those responsible for implementation. Those on the front lines often have the greatest insight into what steps can best enhance the organization’s performance. When performance measures and targets become the exclusive province of the central office, field offices no longer identify with the measures and are not likely to consider them as part of their real world of concerns. The result: performance reporting becomes a compliance exercise and a morale killer.
In these cases, planning offices can
- involve all levels of the organization in the performance measure selection process. A cross-section of employees will be most knowledgeable about the inner workings of programs and how they interact with their operating environment.
- allow field input in establishing national performance targets by setting aside time on the planning calendar.
- help create the environment in which managers are lauded for ambitious measures even if performance is not met because of factors beyond their control.
Plan Time to Plan
It takes time to develop, test, collect, and analyze meaningful performance information. Unfortunately, many agencies do not focus on performance measurement until they are up against budget or other deadlines. Without sufficient time, credibility can suffer and its value to decision makers is reduced. Even if the information is credible, budget decisions may ultimately be based on considerations other than performance. But having little confidence in performance information guarantees limited use in budget negotiations.
Planning offices can
- encourage programs to devote the time and energy necessary to develop valid performance information by facilitating pre-budget planning and priority setting meetings.
- provide the analytical support that will enable programs to calculate the impact of budget changes on performance levels.
- coordinate schedules with the chief operating officer, chief financial officer, and PIO to ensure that all parties are informed and participate in the planning process.
But What Does it Cost?
Do not presume there is a high cost of collecting and reporting on information. Agencies and programs may not give sufficient consideration to the real costs versus benefits of collecting and reporting accurate, reliable, and verifiable performance information. Managers may shy away from using performance measures if they assume that a credible database will have to be built from scratch and they will have to finance this effort.
Expensive performance information may be a show-stopper, but managers also may over-estimate the expense. Several federal programs economize on costs by using existing information sources developed by other agencies to report on performance. For example, both the Food and Drug Administration and Department of Agriculture’s Food Safety and Inspection Service rely on data from the Centers for Disease Control and Prevention to report the incidence of food borne illness.
In today’s budget-constrained environment it is particularly critical for managers to make judicious decisions about their most important performance information needs and select the most cost-effective approach to meeting these needs. To do this, planning offices can
- establish priority needs for performance information in the most critical areas and evaluate the costs and benefits of meeting highest priority needs by working with agency leadership.
- know the real costs of developing and reporting performance measures. In some cases an initial reaction may be that new measures require new sources of information and new expensive data bases. However, performance information may be available from existing sources.
- develop a validation and verification process to ensure the credibility of performance information provided to managers at any level.
Overarching Role of the PIO
One of the key conclusions reached by the recent survey of PIOs was that, on average, only a small percentage of PIO time is devoted to performance management. The PIO plays a critical role in institutionalizing the use of performance measures if three conditions are met:
- An agency must see that using performance information is a day-to-day way of doing business—not just a way to satisfy reporting requirements.
- Performance information must be credible and reliable.
- The PIO must have access to and buy-in from leadership so that they can work as a powerful partnership in implementing a realistic performance improvement initiative.
To help meet these three conditions, PIOs must be dedicated to performance management and be relieved of other collateral duties.
A Science and an Art
To be meaningful to the manager on the ground, performance measures must be considered a critical component of policy debates and strategy implementation. Performance measures should be used as active gauges of strategy effectiveness. Performance measures that are aggregated and reported on a national basis to OMB and Congress keep key stakeholders informed, but to the program managers on the ground they may not have a meaningful connection to the real job of running the program successfully.
Measurement is precise and thrives when definitions are clear, baselines are stable, goals are predictable, and reporting is systematic. Successful performance, on the other hand, is achieved through a dynamic, discontinuous, and often messy process. An astute planning office understands this and treats performance measurement as both science and art. It also recognizes the importance of addressing the dual reporting and managerial responsibilities of performance information.
