Data is everything. Data is the source from which you draw all of your statistics, the foundation upon which you base your facts. Data is there in all of your tables, your spreadsheets, and your system reports. But data does not equal metrics. And, despite a considerable amount of public confusion, metrics are not indicators.
So when we are talking about key performance indicators, what exactly do we mean? Let’s start with a story.
Once there was an airline… Like many in the industry — too many, as the frequent flyers might say — this airline had a lot of problems. Profits were weak, employees were unhappy, the late arrivals rates were high, and customer satisfaction was in the gutter. After some careful thinking they noticed that their main problem was on-time departures. Because the departures were late, the jet spent more time at the gate, which meant paying more fees to the airport. Once in the air, the jets flew faster to try to make up lost time, which burned up more fuel. Despite flying planes the on-time arrival rates were low. Customers missed connections and more baggage was lost, which meant cranky passengers and cranky employees.
Let’s identify the data:
- We have time spent at the gate measured in minutes,
- The departure and arrival times,
- And airport fees.
- We have fuel consumption as a cost.
- We have the number of baggage pieces handled and of those how many arrived on-time.
- Profit margins.
- And, of course, we have the number of passengers and employee and customer satisfaction data.
While data refers to the information that you have, metrics refer to how you approach the data to understand it: e.g., how the data looks over a period of time or the factors that influence the data in some way. Generally speaking, while you do not have control over the available data, you do control what the metrics are — how you evaluate that data. If the data is not reliable and consistent there is very little you can do with it, but once you have a few key data points that you know are reliable, such as the time spent at the gate, you can use that as a metric.
An indicator is one step further — that specific evidence that a certain condition exists or that something that you are trying to achieve has happened or has not happened yet. Moreover, at the highest level, are the key performance indicators — the one, two, or three things that will guide your direction for everything else.
In our story of the airline, for example, we can see that just the failure of one key performance indicator — punctuality, or specifically time spent at the gate in minutes — was directly or indirectly costing the business time, money, and reputation.
Understanding that punctuality was a useful KPI made the airline decide to make important changes in how they got things done. The company decided to invest both in additional personnel for the cleaning crew and in reorganization of the plane fueling schedule to reduce the transition time between plane arrivals and departures. Using this one KPI they were able to turn the entire airline around. Airport fees, fuel costs, lost baggage claims, complaints and employee attrition all went down, while profits and customer satisfaction metrics went up.
What is it about the airline’s understanding of this KPI that holds lessons for the rest of us?
Look at the Right Data
Modern systems mean that massive amounts of information are buried in financial tools, management tools, technical tools, and more. Not all of them will provide useful information; some will even be poor at data integrity. Narrow your field of view to the data most relevant to your particular challenge to focus people on the data points that they need for each step of the change process.
Be Specific About Performance
While on-time delivery is one of the more important performance indicators in many markets, it’s important to think carefully about what that actually means. When you specify delivery in three days time, does that include or exclude the weekends? Is a renegotiated delivery date on a 20-day project a late delivery? Certainly it would be easy to have a high on-time delivery rate if your schedule contains sufficient buffer, so it’s important to consider throughput rates (or units of production per units of time) as at least as important as ratio of on-time deliveries.
Quality is another KPI, and one commonly talked about in the translation and localization industry. We’ve written a lot about localization quality because, as our experience has shown, quality is a subjective moving target, even within the same company. The kind of quality that you may want for your internal documents may be highly different from what you expect for a high-profile marketing campaign, for example.
Whether with time, quality, or any other KPI, it makes good sense (and cents!) to take time to carefully evaluate and delineate what it means to you.
Tailor to Fit Your Own Needs
In my recent seminar on metrics and KPIs, I was asked if there are any standard industry metrics. While there are benefits to industry standards, the answer is still “not really.” Once you get beyond the basics the industry is highly diversified in content types and processes. For example, you can’t measure quality if you can’t quantifiably define it. While there are a number of methodologies for quantitative quality assessment, the classification of what constitutes an error (especially with regards to accuracy and fluency) and how severely it should be weighed, are still very subjective.
For example, quality expectations in Japanese localization demand an approach that is distinct from other major markets. In markets so defined by locale, competitors, suppliers, and more, far more important than standards are developing metrics and revealing KPIs that are specific to the diverse expectations of your target markets.