Data influences the result more than preferences: Some lessons from implementation of multiattribute techniques in a real decision task

摘要

A multiattribute selection task requiring identification of a subset of the best fifteen or so applicants for a faculty position is analyzed with two techniques—SMART and ZAPROS. SMART provides a cardinal measure of each alternative that is easily used to identify the top candidates. ZAPROS relies on ordinal, verbal input from the decision makers, but provides only partial order of alternatives, meaning that the specific number of applicants identified is not guaranteed to be the number desired. Four decision makers took part in the selection task. Essential differences between results across these SMART and ZAPROS were found for all four subjects engaged in this task. Further analysis showed that alternative scores on attributes were found to influence the results more than attribute weights. Verbal scales and judgment used in ZAPROS were considered by the participants to be of more meaning and better enabled understanding of the similarities and differences in preferences and positions. ZAPROS was considered useful in the first stage of the task, as the basis of elaborating group policy in establishing relative importance of attributes and for establishing relative performance of alternatives on attributes.