Multi-Attribute Decision Analysis
Managers who want a structured approach to decision making often turn to Decision Trees, a visual decision support tool that maps out choices and their probable outcomes. Unfortunately, basic decision trees are somewhat limited in their ability to accommodate complex problems with multiple competing decision criteria. For example, a manager deciding between two sustainability projects, say, an energy efficiency enhancement plan and a solar panel installation, would likely first evaluate the electricity savings for the projects. However, there may be other attributes that should also be considered, making the decision more complex. For example, suppose the manager also considers the visibility of the project to be a priority. It then becomes necessary to model both attributes simultaneously in a single decision making model. This would require converting both electricity savings and visibility into a common metric (so as to “compare apples to apples”) and then establishing a tradeoff between the two attributes to arrive at a weighting that reflects the manager’s relative preferences.
The general process for systematically approaching these types of problems is called multi-attribute decision analysis (MADA). The MADA process, in its simplest form consists of four stages:[1]
- Framing of the decision and identification of the goals and objectives to be achieved by the decision maker
- Identification of all decision alternatives and any related attributes that address the decision making objectives
- Specification of preferences, both for each of the individual attributes and between the attributes in the framework
- Ranking of the decision alternatives according to the specified preferences, given the attribute data for each of the alternatives
In the above example, if the manager must decide between several projects, some of which provide substantial electricity savings but less visibility, and others that provide smaller amounts of electricity savings but more visibility, then the preference for electricity savings versus visibility must be considered explicitly. That is, how much electricity savings must a project provide in order for the manager to be willing to settle for a less visible project?
It is important to note that such preferences occur not only between attributes, but also for each attribute. For example, suppose three projects have electricity savings of 15 megawatts (MW), 20 MW, and 25 MW, respectively. Is the incremental value to the decision maker for 20 MW versus 15 MW the same as the incremental value for 25 MW versus 20 MW? Or does the decision maker view 20 MW as being only slightly better than 15 MW, while viewing 25 MW as being significantly better than 20 MW? These are important questions, because once preferences are established, they form the basis for making rational decisions.
Multi-Attribute Utility
As mentioned above, the different attributes in a multi-attribute decision making problem are likely to be measured in different units. Some attributes, such as “visibility” may even be measured on a qualitative scale. The MADA framework makes use of multi-attribute utility theory (MAUT) to both formalize a common units assessment and specify the decision maker’s preferences for each attribute across its respective units scale.[2] The steps in applying MAUT are:
- Defining attributes by which the decision objectives will be measured
- Normalizing the measurement or scale of all attributes across all alternatives
- Weighting the preferences between those attributes
In Step 2, the decision maker must formulate single measure utility functions for each attribute. This utility function mathematically transforms monetary or other values into utility values; so that for every value of an attribute x, there is a corresponding utility value U(x), which is on a standardized scale, such as 0 to 1, or 0 to 100.
For qualitative attributes, we assess utility after establishing an evaluation scale and building the utility function based on that scale. For the example above, the manager might value a moderate amount of visibility only slightly more than little visibility, and only place high value on the highest levels of visibility. In that case, “visibility” would have a convex utility function, as shown in the first curve in Figure 1. Alternately the manager might derive a fair amount of value from even small amounts of visibility, with greater amounts providing diminishing benefit; this would result in a concave utility function, as shown in the second curve in Figure 1. Keeney notes that this process is often as much an art as a science, and that special care should be taken in checking for consistency.[3]
Figure 1 – Example Single Measure Utility Functions
Once the preferences for individual attributes have been specified, the decision maker can then establish preferences between the attributes by specifying the weights in a multi-attribute utility function of the form:
where the k’s are the weights applied to the utility of each attribute, and the sum of the k’s = 1.
Under MAUT, decision models will adhere to a set of basic rules for “clear thinking” that define the structure of decision models and help the decision maker avoid inconsistency. Some example properties of decision models built upon MAUT are:
- Transitivity: decision alternatives can be rank ordered, and that ordering is then transitive (e.g., “I prefer A to B, and B to C, so I will then prefer A to C.”)
- Independence: orderings between two alternatives cannot be altered by a third, unrelated alternative (e.g., “If I prefer A to B, and C is introduced as an alternative, I still prefer A to B.”)
These and other axioms of MAUT also help the decision maker avoid certain paradoxes in their decision policies (e.g., exhibiting risk-averse preferences toward monetary gains but risk-seeking preferences toward monetary losses, as in cases where a decision maker is willing to “throw good money after bad”). The goal of this process is to create a model that will stand up to scrutiny and provide consistent results.
A Case Study: Pepperdine’s Center for Sustainability
Pepperdine’s Center for Sustainability (CFS) presented our recent elective course in advanced decision analysis with an ideal opportunity to apply MADA techniques to help the CFS prioritize a number of competing projects to address their goal of reducing campus-wide electricity consumption by 10 percent. Using principles presented in the course, students applied a seven-step process to model this decision:
- Identify alternatives
- Clarify the goals and objectives, and organize them into a hierarchy
- Identify measures
- Quantify measures for each alternative
- Delineate preferences:
- for attributes
- between attributes
- Rank alternatives
In the following sections, we discuss each of these steps in turn, and describe how we implemented them in Logical Decisions®, a commercially available MADA software application.
1. Identify alternatives: The project alternatives under consideration by the CFS, which were brought to the class to be prioritized and ranked, are shown in Figure 2. Initially these projects had been evaluated only in cost/payoff terms: Capital outlay, ROI, and payback period. However, CFS recognized that the decision included a range of ancillary, if not superseding, non-quantifiable concerns such as regulatory risk, public awareness, and the likelihood of student involvement.
Figure 2 – CFS Project Alternatives
Educational Campaign: Advance the benefits of the initiatives by educating the University community about their energy use.
Energy Policies: Implement policies to change behavior in conjunction with an educational campaign.
Metering Individual Buildings: Install third-party energy use surveillance to fully realize the potential energy savings from individual meters.
Pool Cover: Thermal cover at night on the Raleigh Runnels pool to reduce heating energy required.
Expand Energy Management System (EMS): Save energy by providing schedules to shut-down lighting and HVAC equipment remotely when buildings are unoccupied
Occupancy Sensor Retrofits: Provide an additional layer of savings (over EMS) when a building is occupied but individual areas are unoccupied (i.e., during the work week)
Interior LED Lighting Retrofit: Highly efficient Light Emitting Diode technology
Distributed Chiller Plant: More efficient interconnected system of chillers
Whole Campus Modeling: Model the campus to determine which alternative energy to implement based upon campus needs, parameters, costs, impact, and investment returns
Solar Hot Water: Capture the heat of the sun to heat water for use in showers and pools (option in the residence halls, locker rooms, and on-campus pools)
2. Clarify goals and objectives: The primary goal, “To Reduce Energy Consumption by 10%,” was a mandate given to the CFS, and even though the probable list of alternatives had also already been generated, CFS staff took a “top-down” approach in defining their goals and objectives for ranking the projects. The resulting hierarchy, shown in Figure 3, established four fundamental measures, which roughly correspond to Bryan Norton’s four “Catalog of Sustainability Values.”[4]
Figure 3 – CFS Goals Hierarchy
3. Identify measures: All of the fundamental measures shown in Figure 3 could be further evaluated using multiple sub-measures. For simplicity, the details have been omitted in the Figure, however, as an example, the risk measure was measured in further detail by economic, operational, regulatory, and reputation risk sub-measures.
4. Quantify levels for measures for alternatives: It is preferable to be as objective as possible when quantifying measures (i.e. specific dollar amounts are better than scales, such as “low cost” to “high cost”). For example, we were able to fix kilowatt-hours for electricity conservation. However, specific values were not available for many of our attributes. In such cases, we relied upon appropriate Likert scales (1-5) for our measurement values.
5a. Delineate preferences for individual attributes: Using Logical Decisions®, we were able to assess the single measure utility functions (SUF’s) for each attribute visually with the CFS team, taking special care in determining risk posture for qualitative attributes.
5b. Delineate preferences between attributes: We next assessed tradeoffs between attributes. The most basic method for this is to simply estimate and directly enter the numerical weights for each attribute, making sure that the sum of the weights equals 1. More sophisticated methods are also available, which employ indirect assessments of weights. These methods are intended to remove biases that sometimes occur when the weights are directly estimated.[5] [6] We took that latter approach, using a built-in tradeoff assessment procedure to indirectly assign the weights.
Figure 4 shows the final weightings resulting from this process for the four top-level measures, as displayed by Logical Decisions®, for the CFS project.
Figure 4 – Tradeoff Summary (% Weightings for Measures)
6. Rank Alternatives: Once all the preceding steps have been completed, the software applies the resulting utility functions to the attribute data for each alternative and generates a combined weighted utility score and ranking. In Figure 5 we see that, given the stated preferences, CFS would maximize utility by choosing the Solar Hot Water Heater. This alternative not only provides the highest aggregate utility, by a small margin, but also the most balanced distribution of utility across the CFS’s four top-level measures. The second and third ranked alternatives are separated by a slightly larger margin, mostly determined by the relatively outsized effect of the Risk attributes on the utility of Whole Campus Modeling (WCM) – modeling the campus for potential alternative energy projects.
Figure 5 – Ranking of Sustainability Project Alternatives
We can also see a large discrepancy between the economics of WCM and its limited conservation payoff. The investment cost for WCM is only $45,000, half the cost of the Solar Pool Heater and a fraction of the cost for the Metering Buildings alternative. However, WCM produces no direct conservation.
Results and Reassessment: Before beginning this process, CFS administrators had believed that Metering Buildings would be the “best” of the alternatives, and they were therefore intrigued to find the Solar Heater and Whole Campus Modeling so highly ranked by the decision model. This type of observation is common in MADA applications, and two likely conclusions follow; either 1) the decision maker’s initial intuition was biased, or 2) the decision maker’s preferences have not been accurately assessed. To address the second possibility, CFS was advised that a second iteration with the process would be helpful to validate the preference assessments, or to indicate where they should be revised. In any case, the current results of the MADA process have provided CFS with powerful insights and supporting data for presenting the alternatives for energy conservation to the administration for further consideration.
Conclusions
Given the increasing complexity of decisions facing corporate managers today, managers need a variety of tools to assist them with thinking rationally about their decisions and ultimately to provide support for their decisions. In particular, organizations are seeking a more structured framework for making decisions related to sustainability issues.[7] [8] [9] [10] We have shown how MADA modeling can be a powerful tool for decision makers in those and other cases, offering the following critical functions:
- A process for translating values into decision criteria and/or alternatives, which is a necessary first step in determining exactly what is important to the decision maker
- Indirect approaches for assessing a decision maker’s preferences (e.g., tradeoffs between representative attributes) and developing the resulting attribute weightings, which minimizes the potential for bias
- A methodology for computing rankings, based on the attribute data and the decision maker’s preferences
- Capabilities for sensitivity analysis of results, which makes it possible for a decision maker to validate whether the rankings “make sense,” and if not, to iterate with the process (e.g., go back to re-evaluate preferences).
This article focused on the decision process as managed by a single decision maker. An interesting extension beyond the scope of this paper would be to consider the decision making process as performed by groups, wherein individual preferences between group members would be weighted through the tradeoff process. Furthermore, the CFS goal of reducing electricity by 10 percent was evaluated in isolation in this case. However, other similar sustainability projects will likely arise as part of Pepperdine’s longer-term master plan, and an extension of this work would therefore be to re-evaluate this decision model in the context of that larger plan.
[1] Clemen, R. T. and T. Reilly, Making Hard Decisions with Decision Tools®. Mason, OH: South-Western Cengage Learning, 2001.
[2] Keeney, R. and H. Raiffa, Decisions with Multiple Objectives: Preferences and Value Tradeoffs. New York, NY, USA: Wiley, 1976.
[3] Keeney, R. “Creativity in Decision Making with Value-Focused Thinking.” Sloan Management Review, Summer, (1994): 33-41.
[4] Norton, B. Sustainability: A Philosophy of Adaptive Ecosystem Management. Chicago: University of Chicago Press, 2005.
[5] Hammond, J., R. Keeney, and H. Raiffa. “Even Swaps: A Rational Method for Making Tradeoffs.” Harvard Business Review, March-April, (1998): 137-149.
[6] Keeney, R. “Common Mistakes in Making Value Trade-Offs.” Operations Research, 50 (6), (2002): 935–945.
[7] Wallenius, J., J. S. Dyer, P. C. Fishburn, R. E. Steuer, S. Zionts, and K. Deb. “Multiple Criteria Decision Making, Multiattribute Utility Theory: Recent Accomplishments and What Lies Ahead.” Management Science, 54 (7), (2008): 1336-1349.
[8] Lahdelma, R., P. Salminen, and J. Hokkanen. “Using Multicriteria Methods in Environmental Planning and Management.” Environmental Management, 26 (6), (2000): 595-605.
[9] Miettinen, P., and R. Hämäläinen. “How to benefit from decision analysis in environmental life cycle assessment (LCA).” European Journal of Operational Research, 102 (2), (1997): 279-294.
[10] Rauchsmeyer, F. “Reflections on Ethics and MCA in Environmental Decisions.” Journal of Multi-Criteria Decision Analysis, (2001): 65–74.
