2005 Volume 8 Issue 4

IT MATTERS: Measuring Success

IT MATTERS: Measuring Success

How IT Helps and Hinders the Quest for Value

Where is the life we have lost in living? Where is the wisdom we have lost in knowledge? Where is the knowledge we have lost in information?
T. S. Eliot

Photo: Keith Syvinski

Since the development of the abacus in ancient China, humankind has searched relentlessly for numerical approaches to informed decision making. That progression quickened with the development in 1946 of the world’s first electronic numerical integrator and digital computer (ENIAC),[1] developed by Army Ordnance to compute World War II ballistic firing tables. Many view the ENIAC as the first practical electronic computer. Since then, we have witnessed a steady, exponential increase in computing power and network bandwidth, along with a simultaneous decrease in both size and cost of computing power.

Gordon Moore’s 1965 prediction that computing power would essentially double every 18 to 24 months for the same cost has, astonishingly, held true. Attempting to capitalize on these vast improvements in computer performance, businesses developed powerful decision support systems and collected even more numbers (i.e., data). And now, those information technologies allow business managers to digitize, classify, and evaluate hundreds of strategic variables, providing them with quick and accurate solutions to common business problems.

While this capability is clearly a benefit to decision-makers, it can also create inherent risk namely, that variables easiest to measure and track via information technology and quantitative models can assume greater importance than they deserve. Sometimes, what is critical is also difficult, if not impossible, to measure and, unfortunately, tends not to be considered in the final analysis. In this article we discuss some of the extraordinary benefits that IT and quantitative analysis bring to the decision-making process, and we also challenge the reader to consider, thoughtfully, the inherent risk of reducing complex decisions to measurable, or calculable mathematical models.

How IT and Numbers Can Help Us

Information technologies have, undeniably, had a positive impact on economic growth over the last century. One need look no further than a daily newspaper to see that information technology has radically changed today’s cultures and the way people think. Advanced information systems and networks enable the nearly immediate transmission of digital information in such forms as photographs, streaming video, graphs, numerical indices, financial facts, market analyses, expected values, etc. All of this information flows through and nourishes the global economy as blood does the human body. Numbers, the information systems in which those numbers are stored, and the analytical procedures we use to make sense of those numbers are fundamental to progress in modern societies.

This constant flow of information has at least three distinct advantages for business. It provides 1) an increased awareness of opportunities, 2) an increased awareness of inefficiencies, and 3) an increased ability to predict, monitor, and track performance. Information technologies make it possible for managers to identify opportunities in the form of emerging markets, demand for new products, and new customer service initiatives.

Photo: Lotus Head

For example, premium ice cream company Cold Stone Creamery has a difficult product mix. The staple of their product line is a collection of ice cream dishes recommended by the store. However, by allowing customers to mix and match “add-ins” such as chocolate chips, fresh strawberries, and sour gummy worms into their choice of ice cream flavors, Cold Stone Creamery has an almost limitless set of possible product combinations. Unfortunately, the legacy IT systems at Cold Stone Creamery were not able to record what add-ins were mixed with which flavors. By integrating a new Customer Relationship Management (CRM) system, the company is now able to quickly identify new trends in flavor mixes from their broad base of customers and offer these new flavors as in-store recommendations for new customers. This “sense-and-respond” capability has contributed to Cold Stone Creamery’s dramatic growth from one to 12,000 stores in ten years.[2]

Information Technologies Help Eliminate Inefficiencies, Track Performance

The application of information technologies also permits business organizations to identify and replace established processes that are embedded with inefficiencies. An exciting example of this technology is known as “business intelligence” (BI). BI methodologies make it possible for firms to measure, collect, analyze and act on information on a grand scale. An example of the benefits of BI can be seen at Continental Airlines, whose various BI initiatives have saved the organization more than $500 million over the past six years.[3] Continental’s COO, Larry Kellner, argues that “BI is critical to the accomplishment of our business strategy and has created significant business benefits.”

Information systems such as their “Demand-driven Dispatch” system have helped Continental become a more efficient organization. Prior to developing the Demand–driven Dispatch system, Continental rarely changed flight schedules, plane assignments, or maintenance routines. This inflexibility led to frequent inefficiencies in the form of oversold flights, empty planes and frustrated customers. However, with the Demand-driven Dispatch system, Continental now combines data from revenue forecasts with flight schedule information to identify promising new sources of additional revenue. For instance, the system might recommend larger planes for routes with high demand, or a reduction in the number of flights on unprofitable routes. These systems have helped Continental increase performance since 1996 when they were rated among the “worst” of major U.S. air carriers.

Finally, information technologies have increased a firm’s ability to predict, monitor, and track performance. A powerful example of this capability surfaced in the fall of 2005 as America watched various U.S. agencies forecast the path, strength, and potential for peril of hurricanes Katrina in Louisiana and Mississippi and Rita in Texas. Days before the storms came ashore, analysts were able to assess risks and warn inhabitants in areas that would likely be affected. Such capabilities were unheard of just a decade or two ago.

These advances have been possible only because of the ongoing development and application of IT resources. Undoubtedly business has benefited greatly from rational judgments based on “hard evidence,” i.e. numbers, rather than on “gut reactions”, intuition, or myths.

What Numbers Cannot Measure

When one recognizes the extraordinary value to business of numbers and the information systems that store them, it is easy to see why the maxim at our headiest consulting firms goes something like this: “If it can be performed, it can be measured. If it can be measured, it can be managed.” We ascribe inordinate importance to numbers in our personal and professional lives. Professional presentations usually include charts, summary statistics, financial projections, and expected monetary values. Every day it seems that someone in the media or government proffers a number that will help us better comprehend our world. We saw this recently in the U.S. when the hurricanes threatened. The National Weather Service immediately queried their information systems looking for the numbers that would help them select an optimum strategy.

Our infatuation with numbers has been described most eloquently by Antoine de Saint-Exupéry:

Grown-ups love figures. When you tell them you’ve made a new friend, they never ask you any questions about essential matters. They never say to you “What does his voice sound like? What games does he love best? Does he collect butterflies?” Instead they demand “How old is he? How much does he weigh? How much money does his father make?” Only from these figures do they think they have learned anything about him.[4]

Why then, in spite of all the counting, do we continue to find ourselves disillusioned, confused, and surprised by the “unexpected.” Saint-Exupéry observed that perhaps it is because numbers neither capture everything of importance, nor do they tell the whole story. This is especially true when we insist upon relying on numbers to the exclusion of good judgment or intuition, or when we find that we have asked the wrong questions and therefore have collected the wrong data. In fact, the relative simplicity with which many variables can be identified, measured, and tracked with IT may hinder effective decision making. Decision makers may erroneously choose to focus their attention on the easily measured variables and models rather than giving adequate attention to those elements of a decision that are difficult even impossible to measure. While this decision-making strategy may lead to a mathematically optimal solution, it may not suggest the wise or prudent response.

We described above how Cold Stone Creamery, Continental Airlines, and the National Weather Service use information technologies to identify new opportunities, isolate inefficiencies, and predict, monitor, and track performance. Many companies have followed suit, creating massive data warehouses of customer-based information on what products their customers buy, how often they buy those products, and the stores from which they purchase the products. Yet a true understanding of how their customers came to make those decisions is still anyone’s guess.

Such “measurement above meaning”[5] is not necessarily an asset; neither does it lead to good decision making. As Fortune columnist Michael Schrage argues, “just because single, left-handed, blond customers who drive Volvos purchase 1,450 percent more widgets on alternative Thursdays than do their married, non-blond, right-handed, domestic car-driving counterparts does not a marketing epiphany make.”[6]

Managers must realize that often the number crunching, the cost/benefit analyses, and the net present value calculations can be of great value; but they can also leave us wanting. In the final analysis after the numbers have been counted, the information systems built, the predictions made the ultimate success of a strategic decision may well depend on factors that have escaped measure. For many of the terribly complex problems that managers encounter, cannot be solved using the information from numbers alone. In those circumstances, intuition, profound process knowledge, sound judgment, moral reasoning, and the zeal for managing people will also be essential.


When one considers the tremendous capabilities of information technology, it is easy to see why IT professionals place great value on numbers. Indeed, business strategists from all disciplines, including the authors of this article, have championed this very noble cause.

Encouraged by the many successes, however, we have perhaps put too much emphasis on the value of measurable information. In our haste to measure performance and manage uncertainty, we have been slow to recognize that the information we collect, manipulate, and analyze often has limited value. For example, society is well served by the increases in information technology that enables experts to simulate a variety of hurricane scenarios and prioritize levee construction projects based upon estimates of potential damage.

However, once the simulations are complete and the numbers are “in,” it us up to leaders with strong resolution, sound judgment and moral reasoning to weigh the immeasurable costs of human life and well-being that can be affected by a natural disaster and to make the difficult and morally weighty decisions about which projects to fund. It is difficult when building mathematical models or information technologies, to place value on some very important things the elderly that were left behind, the children separated from their parents, or the untoward effects of pollutants on the environment. Unfortunately, because these “values” can not be easily measured nor included in a simulation model, they tend to be ignored. And, if they are ignored, we not only risk making poor decisions, we also are in danger of creating a world where only those values that can be “seen” will matter. We endorse numbers, and urge our readers to use them.

We also recognize, however, as Peter Drucker points out, tools and concepts are mutually interdependent. That is, the tools we use to analyze our business often force us to see our business differently. As our tools develop and change, so do our businesses.[7] To see rightly, then, we need to look beyond the numbers; calling upon a broader set of values, and making sure that we have asked the right questions, created metrics with meaning, and made judgments that were based on both mathematical and moral reasoning.

[1] Meik, Martin H. (1961) The ENIAC Story – Retrieved from the World Wide Web: 10-6-05 http://ftp.arl.mil/~mike/comphist/eniac-story.html. (no longer accessible)

[2] Schuman, E., (2005). From Gooey Designs to GUI, IT Helps Ice Cream Chain Deliver, CIO Insight, September.

[3] Anderson-Lehman, R., Watson, H.J., Wixom, B., & Hoffer, J. (2004) Continental Airlines Flies High with Real-Time Business Intelligence. MIS Quarterly Executive. 3 (4): 163-176.

[4] Saint-Exupéry, A. (1993). The Little Prince, translated by Katherine Woods. New York, NY: Harcourt Brace Jovanovich.

[5] Schultze, Q. (2002). Habits of the High-Tech Heart, Grand Rapids, MI: Baker Academic Press.

[6] Schrage, M. (1999). “Sixteen Tons of Information Overload,” Fortune, August: 244.

[7] Drucker, P.F. (1998). “The Information Executives Truly Need,” Harvard Business Review, February, 73 (1): 54-62.

Print Friendly, PDF & Email
Authors of the article
Michael L. Williams, PhD, MS
Michael L. Williams, PhD, MS
Michael Williams, PhD, is an assistant professor of Information Systems at the Graziadio School of Business and Management at Pepperdine University. Dr. Williams earned a PhD in Information Systems from the Kelley School of Business at Indiana University. He received an MDiv and an MA from Abilene Christian University. Prior to entering academe, Dr. Williams was an IT consultant in the Washington, D.C. area.
Samuel L. Seaman, PhD
Samuel L. Seaman, PhD, is a professor in the Decision Science discipline in the Graziadio School of Business and Management at Pepperdine University, where he teaches graduate business courses in applied statistics and evidence-based decision analysis. His research and consulting interests generally focus on the application of mathematical models to practical dilemmas in business, health care, and the not-for-profit sector. Whenever possible, he also contemplates the theoretical searching passionately for a linearly optimal solution to the “Particle/Wave Duality Paradox” in his lab at Malibu’s Surfrider Beach
More articles from 2005 Volume 8 Issue 4


For the first time in history, whistleblowers are being recognized as a powerful and positive force for change with new laws protecting the rights of whistleblowers.

Related Articles