The “Measurement Trap”
By William H. Bleuel, PhD, Professor of Decision Sciences
While I strongly agree that measurement in business is important, I also maintain that there is a degree of risk in taking measurements. I refer to this risk as “the measurement trap.” The measurement trap occurs when an executive believes that he can understand the business and customers in his market by increasing the number of instances of measuring aspects of his business and increasing the level of sophistication within his measurement system.
The key to understanding the measurement trap is related to the underlying nature of measurements. In business, we normally assume that a measurement is an accurate assessment of some aspect of our business. But the measurement trap represents a false belief that we can fully understand all aspects of our business strictly through measurement.
First Trap
To understand the subtlety of this trap, consider its most obvious implication—that everything of importance can be measured. This notion suggests that there is no aspect of the business that cannot be measured. While such an idea is a dream of people like me who teach the quantitative aspects of business, it is nevertheless naïve. Can one capture through measurement the essence of the customer interaction or the level of trust that builds between a company and its employees and/or between employees and customers?
To answer such a question, some would offer the employee satisfaction survey and the customer satisfaction survey as evidence that the essence of a successful business enterprise can be measured. To counter this response, consider that satisfaction surveys of new car buyers indicate very high levels of satisfaction, yet the loyalty of car buyers indicated by repeated purchase of the same make vehicle is very low. (The highest such indicator is less than 50 percent.)
Another example of the measurement trap is trying to capture all aspects of employee satisfaction as they relate to customers. For example, consider measures of employee courtesy and professionalism. Each such measure is incomplete, and furthermore is only an indication at best and is misleading at worst. For instance, with rare exception, the employee courtesy measures almost always show high levels of satisfaction among customers, but generally include a bias from the customer that may be based on the fear that if a low score is reported, the service level from the employee may be negatively impacted. On the other hand, the employee may have a good working relationship with repeat customers, so the customer’s evaluation may be biased to “help a friend.” First Trap: All aspects of the business can be measured.
Second Trap
Perhaps one of the most negative aspects of falling into the first measurement trap is that such a belief relieves the executive from the responsibility/necessity of employee interaction. Here the rationale is that all the executive needs to do is review the computer printouts of all the measurements that have been taken.
One of the major influences that led to my understanding the concept of the measurement trap is an executive in the office equipment industry who strongly advocated measurement. This company had a field workforce of repair technicians who repaired equipment on customer sites throughout the United States. The executive spent his day poring over computer printouts, always thinking of new measurements to take. One of his great insights was that the branch service managers were not spending enough time with customers. I was asked to investigate this situation and present solutions for getting the branch service managers more involved with customers. Our analyses revealed some rather dramatic results pertaining to the average service manager:
- The average service managers were signing about 1000 documents per month (time sheets, replacement parts requisitions, etc.).
- The average service manager received about 20,000 pages of computer output per month (parts usage, equipment performance, etc.).
- Each service manager had to inspect the lease cars of every technician twice each year and report such items as worn out wiper blades, burned out light bulbs, and general car cleanliness.
Our findings indicated that the average service manager needed to work overtime just to maintain his paperwork and so had no time available during a standard work week to meet with customers. With this information, we reduced the amount of paperwork at the branch level. The executive, however, believed his own job performance was only related to the computer output on his desk. Second trap: You don’t need to work with people.
Third Trap
A variation on this second trap is a belief that if you do not need to work with people, then you do not have to worry about what kind of people are working for you. If the business is completely measurable, then there is no need to deal with employees or who they are. The people on staff can be the best in the world or a bunch of “goofballs.” The key here is that the manager who believes there is no need for employee interaction may also conclude that the kind of people working for him or her is not important as long as the right measurements are in place.
Such an assumption overlooks the need for employee development. If the employees are incidental and an unimportant aspect of the operation, then is the employee even needed? The executive who is busy spending time behind a closed door analyzing measurements is not developing his replacement and is not giving his direct reports the opportunity to work with him and learn more about the business in order to become more valuable employees. Third trap: Company culture is not important.
Fourth Trap
Once an executive is caught in the measurement trap, the largest impact on the organization is implementation of the measurement trap. When an executive believes that the business can be defined by measurements and then a problem occurs, the executive concludes that more measurements are needed to analyze and solve the problem. This spiral continues when the additional measurements do not provide the solution to the problem, and so the executive begins to request further measurements.
In the extreme case, this continued search for the necessary measurements to solve a problem can impact the business to a point where the cost of measurement exceeds the cost of the problem, a sequence that may lead to “paralysis by analysis.” This flaw perpetuates the measurement trap due to the notions that with sufficient information, all business problems can be analyzed and that lack of available information merely indicates that more information must be gathered. Fourth trap: Given enough measurements, all problems can be analyzed.
Avoiding the Measurement Trap
What do you do to protect yourself from getting caught in the measurement trap, and how do you get out of it?
- Examine each current measurement and verify that it is necessary. Many companies take unnecessary measurements that are costly and have little or no impact on the successful operation of the business. (Consider the executive at the office equipment company noted previously. Was an examination of the wiper blades a good use of the service manager’s time?) Managers should take note of how much data the company already has and whether it is valuable information.
- Use measurements judiciously. Select measurements carefully and constantly audit the results for accuracy. While too many measurements can be expensive, time consuming, and misleading, some degree of measurement is necessary to successfully run a business. However, remember that measurements are like looking out the rear view mirror; they only tell you what has happened, not necessarily what is going to happen.
- Assess the cost of measurement systems. If the costs of measurement are disproportionate to the level of your organization’s operation, you are probably taking too many measurements and are therefore probably heading into the measurement trap. When looking at the cost of measurement, remember to include the costs of data collection and assessment.
Ending the Measurement Trap
The first step to ending the measurement trap is to recognize that you are in it. After reading the four measurement traps, if you believe you are caught, you have already taken the first step.
The second step is to change your business from an internal perspective based on measurement to an external perspective based on interaction with employees and customers.
The final step is to remember that measurements can be valuable and should be used where necessary, but measurement in and of itself will not provide the total answer to any question.
