Improving Organizational Performance
Through the Use of Measures
One consulting service we provide for many clients is metrics development. These projects are always a great opportunity for us to help our clients focus on mission outcomes and work with the end in mind. This type of engagement aligns directly with the CGS mission of helping our clients deliver on their public promise.
In these engagements, we begin by understanding what outcomes our client organization is looking to achieve and then identify what processes are operating to deliver on those mission outcomes. Then we examine what process-level or activity-level metrics are in place, as well as how effectively existing measures are currently measuring mission outcomes.
In the press of operations, the view of the forest as a whole (the mission) is often lost by the distraction of the trees (activities). A warning symptom that this may be happening is that a “performance” report is reporting activity-level measures with little or no context. Although it is clear the activities being reported are important, it is not clear what the performance of those activities represent in terms of mission delivery.
Another typical finding is that the metrics being reported almost always achieve a nearly perfect result. While this view demonstrates that the organization is “doing its job,” the metrics are of little use for improving service delivery. For example, a metric such us “we will complete 100% of our tasks within 90 days” (often a legislated time frame), leads to counting how many things were completed within 90 days. This leaves many unanswered questions, such as:
■ How many things had to be done?
■ What’s the base rate?
■ What’s the backlog? Is it growing or shrinking?
■ Could we get this done on average in 60 days?
Such metrics make it difficult to tell if an organization is doing better or worse over time, whether it needs more or less resources to do the work in a timely manner, or whether or not program costs or other indicators of efficiency or effectiveness are on target. Metrics need variance in order to differentiate between typical and less typical performance. One way to address this challenge and improve measurement is to move from counting things to looking at things as a percent of the whole. For example, in administrative law processes, the percent of open cases completed in a month is an improvement over the number of cases closed in a month. Making this shift changes the measure from an activity (case processing) to an output (percent of cases closed). Reporting on this over time, allows an organization to observe program performance trends such as if its backlog is growing or shrinking, if there are seasonal influences, if staffing meets demand (e.g. caseload per attorney in order to reduce the backlog), etc. While measuring an output is an improvement, it is not the same as measuring an outcome.
If the program’s mission is to process appeals within a regulatory time frame, in order to fairly provide for a public good or service, then the percent of cases closed is important, but insufficient to know if the program is meeting its outcome. One approach to improve output metrics and move them in the direction of outcomes is to take the following steps:
1) Measure the output as a part of the whole (appeals as a part of all claims, or claims as a part of service provision, or beneficiaries reached as a percentage of all beneficiaries served),
2) Display the measure over time to turn it into a process measure, and
3) Consider ways to display the metric to reduce unusual peaks or valleys caused by unique circumstances. This is often done by using moving averages or other smoothing algorithms.
The next time the topic of program performance arises, consider assessing your measures to see if they are focused on inputs, activities, outputs or outcomes (key perspectives in logic models and process flow charts). Given the information and resources available to the organization, find the most appropriate balance of activity, output and outcome measures to better understand progress over time towards achieving the mission.
