journal article·2021-11-05Open access

Methods for Weighting Decisions to Assist Modelers and Decision Analysts: A Review of Ratio Assignment and Approximate Techniques

Barry Ezell, Christopher Lynch, Patrick Hester

Applied Sciences· Volume 11· Issue 21

Abstract

Computational models and simulations often involve representations of decision-making processes. Numerous methods exist for representing decision-making at varied resolution levels based on the objectives of the simulation and the desired level of fidelity for validation. Decision making relies on the type of decision and the criteria that is appropriate for making the decision; therefore, decision makers can reach unique decisions that meet their own needs given the same information. Accounting for personalized weighting scales can help to reflect a more realistic state for a modeled system. To this end, this article reviews and summarizes eight multi-criteria decision analysis (MCDA) techniques that serve as options for reaching unique decisions based on personally and individually ranked criteria. These techniques are organized into a taxonomy of ratio assignment and approximate techniques, and the strengths and limitations of each are explored. We compare these techniques potential uses across the Agent-Based Modeling (ABM), System Dynamics (SD), and Discrete Event Simulation (DES) modeling paradigms to inform current researchers, students, and practitioners on the state-of-the-art and to enable new researchers to utilize methods for modeling multi-criteria decisions.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Complex Systems and Decision Making · Cognitive Science and Mapping · Innovation Diffusion and Forecasting

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Virginia Modeling, Analysis and Simulation Center, Old Dominion University, Suffolk, VA 23435, USA

Modus Operandi Inc., Melbourne, FL 32901, USA

References (100+)

Show source-provided references
1. Simulation and Reality: The Big Picture10.1142/S1793962310000079 ↗
2. Zeigler, B.P., Prähofer, H., and Kim, T.G. (2000). Theory of Modeling and Simulation: Integrating Discrete Event and Continuous Complex Dynamic Systems, Academic Press. [2nd ed.].
3. Verification and Validation of Simulation Models10.1057/jos.2012.20 ↗
4. Model Base Management for Multifacetted Systems10.1145/140765.140791 ↗
5. On the Need for Contextualized Introspective Models to Improve Reuse and Composability of Defense Simulations10.1177/875647930400100302 ↗
6. Spiegel, M., Reynolds, P.F., and Brogan, D.C. (2005, January 4). A Case Study of Model Context for Simulation Composability and Reusability. Proceedings of the 2005 Winter Simulation Conference, Orlando, FL, USA.
7. Casilimas, L., Corrales, D.C., Solarte Montoya, M., Rahn, E., Robin, M.-H., Aubertot, J.-N., and Corrales, J.C. (2021). HMP-Coffee: A Hierarchical Multicriteria Model to Estimate the Profitability for Small Coffee Farming in Colombia. Appl. Sci., 11.10.3390/app11156880 ↗
8. Lynch, C.J. (2014). A Multi-Paradigm Modeling Framework for Modeling and Simulating Problem Situations. [Master’s Thesis, Old Dominion University].10.1109/WSC.2014.7020019 ↗
9. Group Model-Building: Tackling Messy Problems10.1002/(SICI)1099-1727(199924)15:4<379::AID-SDR179>3.0.CO;2-E ↗
10. Fernández, E., Rangel-Valdez, N., Cruz-Reyes, L., and Gomez-Santillan, C. (2021). A New Approach to Group Multi-Objective Optimization under Imperfect Information and Its Application to Project Portfolio Optimization. Appl. Sci., 11.10.3390/app11104575 ↗
11. Barry, P., and Koehler, M. (2004, January 5–8). Simulation in Context: Using Data Farming for Decision Support. Proceedings of the 2004 Winter Simulation Conference, Washington, DC, USA.
12. Keeney, R.L., and Raiffa, H.G. (1976). Decisions with Multiple Objectives: Preferences and Value Tradeoffs, Wiley & Sons.
13. Multi-criteria decision analysis in natural resource management: A critical review of methods and new modelling paradigms10.1016/j.foreco.2006.03.023 ↗
14. Aenishaenslin, C., Gern, L., Michel, P., Ravel, A., Hongoh, V., Waaub, J.-P., Milord, F., and Bélanger, D. (2015). Adaptation and evaluation of a multi-criteria decision analysis model for Lyme disease prevention. PLoS ONE, 10.10.1371/journal.pone.0135171 ↗
15. Hongoh, V., Campagna, C., Panic, M., Samuel, O., Gosselin, P., Waaub, J.-P., Ravel, A., Samoura, K., and Michel, P. (2016). Assessing interventions to manage West Nile virus using multi-criteria decision analysis with risk scenarios. PLoS ONE, 11.10.1371/journal.pone.0160651 ↗
16. Scholten, L., Maurer, M., and Lienert, J. (2017). Comparing multi-criteria decision analysis and integrated assessment to support long-term water supply planning. PLoS ONE, 12.10.1371/journal.pone.0176663 ↗
17. Infrastructure Vulnerability Assessment Model (I-VAM)10.1111/j.1539-6924.2007.00907.x ↗
18. An Improvement Selection Methodology for Key Performance Indicators10.1007/s10669-016-9591-8 ↗
19. Homeland Security and Emergency Management Grant Allocation
20. Prioritizing Countries by Concern Regarding Access to Weapons of Mass Destruction Materials
21. Modeling managerial behavior: Misperceptions of feedback in a dynamic decision making experiment10.1287/mnsc.35.3.321 ↗
22. Forrester, J.W. (1961). Industrial Dynamics, The MIT Press.
23. Discrete-event simulation: From the pioneers to the present, what next?10.1057/palgrave.jors.2601864 ↗
24. Discrete Event Simulation for Healthcare Organizations: A Tool for Decision Making
25. Padilla, J.J., Lynch, C.J., Kavak, H., Diallo, S.Y., Gore, R., Barraco, A., and Jenkins, B. (2016, January 11–14). Using Simulation Games for Teaching and Learning Discrete-Event Simulation. Proceedings of the 2016 Winter Simulation Conference, Arlington, VA, USA.10.1109/WSC.2016.7822368 ↗
26. Kelton, W.D., Sadowski, R.P., and Swets, N.B. (2010). Simulation with Arena, McGraw-Hill. [5th ed.].
27. Agent-Based Computational Models and Generative Social Science10.1002/(SICI)1099-0526(199905/06)4:5<41::AID-CPLX9>3.0.CO;2-F ↗
28. Gilbert, N. (2008). Using Agent-Based Models in Social Science Research. Agent-Based Models, Sage.10.4135/9781412983259 ↗
29. Epstein, J.M., and Axtell, R. (1996). Growing Artificial Societies: Social Science from the Bottom Up, The MIT Press.10.7551/mitpress/3374.001.0001 ↗
30. Dynamic Models of Segregation10.1080/0022250X.1971.9989794 ↗
31. Simulating Macro-Level Effects from Micro-Level Observations10.1287/mnsc.2017.2877 ↗
32. Wooldridge, M., and Jennings, N.R. (1994). Agent Theories, Architectures, and Languages: A Survey. Intelligent Agents ATAL, Springer.10.1007/3-540-58855-8 ↗
33. Lynch, C.J., Diallo, S.Y., and Tolk, A. (2013, January 7–10). Representing the Ballistic Missile Defense System using Agent-Based Modeling. Proceedings of the 2013 Spring Simulation Multi-Conference-Military Modeling & Simulation Symposium, San Diego, CA, USA.
34. A Generative Model of the Mutual Escalation of Anxiety Between Religious Groups10.18564/jasss.3840 ↗
35. Wooldridge, M., and Fisher, M. (1994). A Decision Procedure for a Temporal Belief Logic. Temporal Logic ICTL 1994, Springer.10.1007/BFb0013996 ↗
36. BehavDT: A Behavioral Decision Tree Learning to Build User-Centric Context-Aware Predictive Model10.1007/s11036-019-01443-z ↗
37. Ching, W.-K., Huang, X., Ng, M.K., and Siu, T.-K. (2013). Markov Chains: Models, Algorithms and Applications, Springer. [2nd ed.].10.1007/978-1-4614-6312-2 ↗
38. Razzaq, M., and Ahmad, J. (2015). Petri Net and Probabilistic Model Checking Based Approach for the Modelling, Simulation and Verification of Internet Worm Propagation. PLoS ONE, 10.10.1371/journal.pone.0145690 ↗
39. Sokolowski, J.A., and Banks, C.M. (2010). Modeling and Simulation Fundamentals: Theoretical Underpinnings and Practical Domains, John Wiley & Sons.10.1002/9780470590621 ↗
40. Linear models in decision making10.1037/h0037613 ↗
41. Enhanced decision modeling using multiagent system simulation10.1177/0037549703038886 ↗
42. Links between systems thinking and complex decision making10.1002/sdr.281 ↗
43. How do agents make decisions? A survey10.18564/jasss.2687 ↗
44. Jin, H., and Goodrum, P.M. (2021). Optimal Fall Protection System Selection Using a Fuzzy Multi-Criteria Decision-Making Approach for Construction Sites. Appl. Sci., 11.10.3390/app11115296 ↗
45. Kim, B.-S., Shah, B., Al-Obediat, F., Ullah, S., Kim, K.H., and Kim, K.-I. (2018). An enhanced mobility and temperature aware routing protocol through multi-criteria decision making method in wireless body area networks. Appl. Sci., 8.10.3390/app8112245 ↗
46. García, V., Sánchez, J.S., and Marqués, A.I. (2019). Synergetic application of multi-criteria decision-making models to credit granting decision problems. Appl. Sci., 9.10.3390/app9235052 ↗
47. Urbaniak, K., Wątróbski, J., and Sałabun, W. (2020). Identification of Players Ranking in E-Sport. Appl. Sci., 10.10.3390/app10196768 ↗
48. Panapakidis, I.P., and Christoforidis, G.C. (2018). Optimal selection of clustering algorithm via Multi-Criteria Decision Analysis (MCDA) for load profiling applications. Appl. Sci., 8.10.3390/app8020237 ↗
49. Shaikh, S.A., Memon, M., and Kim, K.-S. (2021). A Multi-Criteria Decision-Making Approach for Ideal Business Location Identification. Appl. Sci., 11.10.3390/app11114983 ↗
50. Clemente-Suárez, V.J., Navarro-Jiménez, E., Ruisoto, P., Dalamitros, A.A., Beltran-Velasco, A.I., Hormeño-Holgado, A., Laborde-Cárdenas, C.C., and Tornero-Aguilera, J.F. (2021). Performance of Fuzzy Multi-Criteria Decision Analysis of Emergency System in COVID-19 Pandemic. An Extensive Narrative Review. Int. J. Environ. Res. Public Health, 18.10.3390/ijerph18105208 ↗
51. Ranking Range Based Approach to MADM under Incomplete Context and its Application in Venture Investment Evaluation10.3846/tede.2019.10296 ↗
52. Xiao, J., Wang, X., and Zhang, H. (2021). Exploring the Ordinal Classifications of Failure Modes in the Reliability Management: An Optimization-Based Consensus Model with Bounded Confidences. Group Decis. Negot., 1–32.10.1007/s10726-021-09756-9 ↗
53. An Overview on Feedback Mechanisms with Minimum Adjustment or Cost in Consensus Reaching in Group Decision Making: Research Paradigms and Challenges10.1016/j.inffus.2020.03.001 ↗
54. Systemic Analysis of a Drug Trafficking Mess10.1504/IJSSE.2019.102837 ↗
55. Multi-Sensor Remote Sensing for Drought Characterization: Current Status, Opportunities and a Roadmap for the Future10.1016/j.rse.2021.112313 ↗
56. Multiplicative Utility Functions10.1287/opre.22.1.22 ↗
57. Hit-and-Run Enables Efficient Weight Generation for Simulation-based Multiple Criteria Decision Analysis10.1016/j.ejor.2012.08.026 ↗
58. Multi-Attribute Decision Making: A Simulation Comparison of Select Methods10.1016/S0377-2217(97)00147-1 ↗
59. The effect of attribute ranges on weights in multiattribute utility measurements
60. Comparison of weighting judgments in multiattribute utility measurement10.1287/mnsc.37.12.1603 ↗
61. A comparison of weight approximation techniques in multiattribute utility decision making10.1016/0030-5073(81)90015-5 ↗
62. Behavioral and procedural consequences of structural variation in value trees10.1016/S0377-2217(00)00255-1 ↗
63. The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capability for Processing Information10.1037/h0043158 ↗
64. Comparing hierarchical and non-hierarchical weighting methods for eliciting multiattribute value models10.1287/mnsc.33.4.442 ↗
65. Pöyhönen, M. (1998). On Attribute Weighting in Value Trees. [Ph.D. Thesis, Helsinki University of Technology].10.1002/(SICI)1099-0771(199806)11:2<139::AID-BDM293>3.3.CO;2-D ↗
66. Interpretation of criteria weights in multicriteria decision making10.1016/S0360-8352(00)00019-X ↗
67. Range sensitivity of attribute weights in multiattribute value models10.1006/obhd.1995.1048 ↗
68. Behavioral Issues in MCDM: Neglected research questions10.1002/(SICI)1099-1360(199609)5:3<178::AID-MCDA123>3.0.CO;2-P ↗
69. On a short-coming of Saaty’s method of analytic hierarchies10.1016/0305-0483(83)90047-6 ↗
70. On the measurement of preferences in the Analytic Hierarchy Process10.1002/(SICI)1099-1360(199711)6:6<309::AID-MCDA163>3.0.CO;2-2 ↗
71. How to use multiattribute utility measurement for social decisionmaking10.1109/TSMC.1977.4309720 ↗
72. Von Winterfeldt, D., and Edwards, W. (1986). Decision Analysis and Behavioral Research, Cambridge University Press.
73. SMARTS and SMARTER: Improved simple methods for multiattribute utility measurement10.1006/obhd.1994.1087 ↗
74. Saaty, T.L. (1980). The Analytic Hierarchy Process, McGraw Hill.10.21236/ADA214804 ↗
75. Multiple Criteria Decision Making, Multiattribute Utility Theory: Recent Accomplishments and What Lies Ahead
76. An analysis of multi-criteria decision making methods
77. Remarks on the Analytic Hierarchy Process10.1287/mnsc.36.3.249 ↗
78. Attribute weighting methods and decision quality in the presence of response error: A simulation study10.1002/(SICI)1099-0771(199806)11:2<85::AID-BDM282>3.0.CO;2-K ↗
79. Kapur, J.N. (2009). Maximum Entropy Principles in Science and Engineering, New Age.
80. Decision quality using ranked attribute weights10.1287/mnsc.42.11.1515 ↗
81. U.S. Coast Guard (1994). Coast Guard Process Improvement Guide: Total Quality Tools for Teams and Individuals.
82. Lynch, C.J., Diallo, S.Y., Kavak, H., and Padilla, J.J. (2020). A Content Analysis-based Approach to Explore Simulation Verification and Identify its Current Challenges. PLoS ONE, 15.10.1371/journal.pone.0232929 ↗
83. Formal Methods, Statistical Debugging and Exploratory Analysis in Support of System Development: Towards a Verification and Validation Calculator Tool10.1142/S1793962316410014 ↗
84. Advancing the Art of Simulation in the Social Sciences10.1002/(SICI)1099-0526(199711/12)3:2<16::AID-CPLX4>3.0.CO;2-K ↗
85. Deterministic chaos in models of human behavior: Methodological issues and experimental results10.1002/sdr.4260040109 ↗
86. Insight Maker: A General-Purpose Tool for Web-based Modeling & Simulation10.1016/j.simpat.2014.03.013 ↗
87. Padilla, J.J., Diallo, S.Y., Barraco, A., Kavak, H., and Lynch, C.J. (2014, January 7–10). Cloud-Based Simulators: Making Simulations Accessible to Non-Experts and Experts Alike. Proceedings of the 2014 Winter Simulation Conference, Savanah, GA, USA.10.1109/WSC.2014.7020192 ↗
88. Lynch, C.J., Padilla, J.J., Diallo, S.Y., Sokolowski, J.A., and Banks, C.M. (2014, January 7–10). A Multi-Paradigm Modeling Framework for Modeling and Simulating Problem Situations. Proceedings of the 2014 Winter Simulation Conference, Savanah, GA, USA.10.1109/WSC.2014.7020019 ↗
89. Lynch, C.J., and Diallo, S.Y. (2015, January 6–9). A Taxonomy for Classifying Terminologies that Describe Simulations with Multiple Models. Proceedings of the 2015 Winter Simulation Conference, Huntington Beach, CA, USA.10.1109/WSC.2015.7408282 ↗
90. Reference Modelling in Support of M&S—Foundations and Applications10.1057/jos.2013.3 ↗
91. MacKenzie, G.R., Schulmeyer, G.G., and Yilmaz, L. (2002, January 22–24). Verification technology potential with different modeling and simulation development and implementation paradigms. Proceedings of the Foundations for V&V in the 21st Century Workshop, Laurel, MD, USA.
92. Eldabi, T., Balaban, M., Brailsford, S., Mustafee, N., Nance, R.E., Onggo, B.S., and Sargent, R. (2016, January 11–14). Hybrid Simulation: Historical Lessons, Present Challenges and Futures. Proceedings of the 2016 Winter Simulation Conference, Arlington, VA, USA.10.1109/WSC.2016.7822192 ↗
93. Vangheluwe, H., De Lara, J., and Mosterman, P.J. (2002, January 7–10). An Introduction to Multi-Paradigm Modelling and Simulation. Proceedings of the AIS’2002 Conference (AI, Simulation and Planning in High Autonomy Systems), Lisboa, Portugal.
94. Balaban, M., Hester, P., and Diallo, S. (2014, January 7–10). Towards a Theory of Multi-Method M&S Approach: Part I. Proceedings of the 2014 Winter Simulation Conference, Savanah, GA, USA.10.1109/WSC.2014.7020016 ↗
95. Agent-based modeling: Methods and techniques for simulating human systems10.1073/pnas.082080899 ↗
96. Epstein, J.M. (2014). Agent_Zero: Toward Neurocognitive Foundations for Generative Social Science, Princeton University Press.10.23943/princeton/9780691158884.001.0001 ↗
97. Modelling terror management theory: Computer simulations of the impact of mortality salience on religiosity10.1080/2153599X.2016.1238846 ↗
98. A network agent-based model of ethnocentrism and intergroup cooperation10.1007/s11135-019-00856-y ↗
99. Enhancing recycling of construction materials: An agent based model with empirically based decision parameters10.18564/jasss.2528 ↗
100. An evolutionary approach to norms10.2307/1960858 ↗