Rationalizing Decisions

[L.J.Steele]

This may have relevance to other subconscious bias issues, and helping folks understand why they make the choices they do.

How people make up good reasons for bad behaviour


YOU are deciding between two magazines to read. The one you choose just happens to feature photos of women in very small swimsuits. But you do not, you claim, pick that particular magazine for the bathing beauties; it happens to have more interesting articles, or better coverage of copper mining in the Democratic Republic of the Congo. You will say this even in the midst of a lab experiment that has been set up so that the only possible difference between the two magazines is the presence (or absence) of swimsuits.

Such was the finding of Zoë Chance, a doctoral student, and Michael Norton, a marketing professor, both at Harvard Business School. The pair were investigating how people justify “questionable” behaviour (Mr Norton’s word) to themselves after the fact. They asked 23 male students to choose between two sports magazines, one with broader coverage and one with more feature articles. The magazine which also happened to contain a special swimsuit issue was picked three-quarters of the time, regardless of the other content. But asked why they chose that particular magazine, the subjects pointed to either the sports coverage or the greater number of features—whichever happened to accompany the bikinis.

http://www.economist.com/business-educa ... d=14739888

[Charles Parker]

There are a couple of things in the 2nd article that I think might be a key to a better understanding of the article and maybe the study itself.

Because people do not want to be perceived as (or feel) unethical or immoral, they make excuses for their shameful behavior—even to themselves.

Zoe Chance is a female

Perhaps the 23 males to hide the perception of immoral or shameful behavior answered that way when the female researcher asked them. If the researcher asking the questions was male would the results of the study be different. I would think it would be real hard to get gender bias out of that type of study (not impossible but difficult).

I am glad you posted this Ms Steele. For several months I have been reading and studying "The Art of Critical Decision Making" and "With Good Reason: An Introduction to Informal Fallacies". I have been thinking about posting some things on critical reasoning and decision making. Even though I have just started it has opened my eyes and ears to a number of the arguments that are being presented here now and then.

Here are some things from 'The Art of Critical Decision Making' that I believe are key to some issues on this forum----not just with practioneers but with the critics as well.

Affective Conflict: Disagreement that is rooted in personality clashes and personal friction. It involves emotion and anger. It is not an issue or task oriented in nature.

Anchoring Bias: Refers to the notion that we sometimes allow an initial reference point to distort our estimates (sounds familiar to me)

Groupthink: The term coined by social psychologist Irving Janis to describe the powerful social pressures for conformity that sometimes arise within groups, causing people to self-censor their views---particularly dissenting opinions.

Intuition: Fundamentally a process of pattern recognition based on past experience. Through pattern matching, individuals are able to make intuitive judgements without going through an appraisal of multiple alternatives, as many 'rational' models of choice would suggest.

Practical Drift: The term coined by Scott Snook to describe how accepted, taken-for-granted practice within organizations gradually can move away from standard operating procedure.

Prospect Theory: Put forth by Amos Tversky and Daniel Kahneman, this theory argues that individuals will exhibit different risk-taking tendencies depending on how a decision is framed. (This one is very interesting and I think may have some application in my line of work)

Recency Effect: The tendency to overweight readily available information, specifically recent data, when judging the probability of certain events occurring in the future. (I think this one could have application to some of the critics and possibly the NAS investigation)

More on decision making and reasoning later.

[Kathleen Birnbaum]

For the sake of discussion and/or argument - is it not a form of Bias to decide they really weren't picking the magazine for the very reasons they gave? They may have indeed given the truthful answer of sports coverage or greater number of features and the bikinis just happened to be there also. It seems to me that Bias can be perceived by the very act of not accepting their answers as such.

[Boyd Baumgartner]

Charles,
If you like Tversky, "Judgement Under Uncertainty: heuristics and biases" is a must.
http://books.google.com/books?id=_0H8gwj4a1MC&printsec=frontcover&dq=judgment+under+uncertainty&ei=2a0CS8u4B4qQlQT-z5miDw#v=onepage&q=&f=false (limited preview)

Also, some other topics you may find interesting are

Cluster illusion
Availability heuristic

I look forward to a post on reasoning and fallacy in the fingerprint realm.

[L.J.Steele]

For the sake of discussion and/or argument - is it not a form of Bias to decide they really weren't picking the magazine for the very reasons they gave? They may have indeed given the truthful answer of sports coverage or greater number of features and the bikinis just happened to be there also. It seems to me that Bias can be perceived by the very act of not accepting their answers as such.

The full study is here
http://hbswk.hbs.edu/item/6283.html

"We asked 23 male participants (Mage = 20.9) to complete this experiment as part
of a class requirement. We told participants we were interested in the criteria they
thought were important in choosing magazines, and introduced two sports magazines.
Both had won the same number of Associated Press Journalism Awards, and had similar
average issue lengths.

We manipulated two attributes of the magazines, such that each magazine
dominated on one attribute. One magazine had a higher number of sports covered per
issue than the other (9 vs. 6), while also having a lower average number of feature articles
per issue (12 vs. 19). In addition, each magazine was advertised as having one special issue: either a Swimsuit Issue (a questionable preference) or a “Year’s Top 10 Athletes”
special issue. Most importantly, we varied which magazine came with the Swimsuit
Issue; for half of our participants, it accompanied the magazine with more sports, while
for the other half, it accompanied the magazine with more feature articles.

We expected our male participants to select the magazine subscription with the
Swimsuit Issue regardless of whether it covered more sports or contained more articles,
and then, in an effort to justify their questionable behavior, to inflate the value of the
attribute favoring that magazine – either the number of sports covered or the number of
articles per issue.

Participants examined the descriptions of the two magazines, circled the magazine
they would choose, and then ranked criteria (average issue length, number of awards,
annual special issues, number of sports covered, average number of articles, and “other”)
in terms of how important they were in their decision. Overall, and as expected,
participants overwhelmingly picked the magazine with the Swimsuit Issue (74%), χ2 (1) =
5.26, p < .03. While 92% of participants selected the magazine with more articles when
that magazine was paired with the Swimsuit Issue, only 46% picked this magazine when
it did not have the Swimsuit Issue paired with it, meaning that 54% of participants
suddenly preferred the magazine with more sports covered, χ2 (1) = 5.79, p < .02, which
just happened to include the Swimsuit Issue (see Table 1)."

They've got a pretty good argument that, consciously or not, the subjects change their mind about what's important (or what they say is important) to get the magainze with the swimsuit issue. The maganitute of the change is pretty striking.

[Charles Parker]

Ms Steele, although they do mention bias twice in the 23 pages (once having to do with race), but IMHO and with all due respect I still believe it is a study that people do not want or like to be perceived as unethical or immoral, that people make excuses for shameful behavior or as the researchers say poor decisions.

I am really not catching on to the bias connection.

I did like one of their first quotes tho:

He uses statistics as a drunken man uses lamp-posts – for support rather than illumination.
Andrew Lang (1844-1912)

I think I will hang on to that one.

[L.J.Steele]

all due respect I still believe it is a study that people do not want or like to be perceived as unethical or immoral, that people make excuses for shameful behavior or as the researchers say poor decisions.

I think the question becomes to what extent are the people making excuses to themselves. If a fellow picking the magazine with the swimsuit issue consciously knows that's why he wants it, and consciously gives a false excuse for doing so, that's one thing. If a fellow is making these decisions subconsciously, and changes the weight he gives to the two variables without knowing that's what's happened, that's a different problem. I'm certainly not saying this is definative, just interesting.

The relevance to the context/confirmation bias issue would be that second problem. Let us assume that Examiner X has a difficult latent print. There's a feature that could be an explainable distortion or it could be a dissimilarity which would result in exclusion. He or she has to make that decision. I think this study is helpful in thinking about the subconscious processes that might underlie the conclusion and explanation for it.

[Gerald Clough]

What's interesting is that subjects in one cited study involving choices of movie to watch did not simply decide they would rather sit with the attractive woman and avoid the disabled person and perception be damned. In the reported study in this paper, they did not simply indicate by rating "Other" or "Special Issues" higher that they simply preferred magazines with images of swimsuit models. In one of the cited studies, subjects' relative analysis of the values of persons of difference races could be correctly predicted by their scores on a prejudice measure. Those prejudice scores would not, I suspect, have surprised those subjects. The problem with all these studies is that they do not study natural behavior. They study the behavior of test subjects. They may not know the nature of the study, but they are aware that they are being observed, and not casually observed. Being under observation is a strong motivator to justify choices or to avoid desires that cannot be easily justified to a known observer.

Subscribers to Playboy know why they subscribe. "I subscribe for the articles" does not reflect a studied choice. It is a statement made to avoid censure. Subscribers choosing sports magazines for the swimsuit issue do not imagine that they select for the articles. Those selecting candidates for race or gender are not fooling themselves. The male subjects are quite aware that they would prefer to sit with the attractive female, and they clearly will take advantage of a perceived acceptable excuse to do so - in the study. I an actual unobserved choice, they may well simply choose the most valued motivator, a distinct preference for one movie over another or a frank choice of companion. Under natural conditions, a male among males will likely frankly say, "Hey, toss me the swimsuit issue."

I would say that bias can only be cited if the evaluations of criteria are subconsciously altered by a desired selection. And I'm not at all convinced, frankly doubt, actually, that the kind of unconscious alteration they presume is what's happening in the study. Rather, it could as well be plain choice and attempted justification. The equivalent in our field would be deliberate attempt to produce a desired outcome and exaggerated or minimized valuing of what's observed, which isn't at all a bias issue. Bias would be an unconscious prejudice in favor of one outcome at influences perception, and in these studies, I don't think the desired outcome is anything but a conscious desire.

I think there tends to be some researcher bias in these things, because they seem to presume that their studies show that all this is subconsciously internal, because that's what would make it most interesting to folks concerned with how behaviors are manipulated. There is some real application to our field, but I think it's on a different plane from actual procedural issues and applies more to how we as a community deal with discussions of things like perceived sufficiency. And it is historically difficult to extrapolate from general studies to the specifics of a particular practice. Legally, it has generally been required that studies be specific to a given practice, an example being early attempts to present general perception studies to eyewitness issues where only relatively recently some expertise gained from very focused studies began to get into court.

[g.]

Good discussion on this thread. The relevant question for me is whether or not the effect in Ms Steele's bias study is actually present during fingerprint exams. For instance, if a colleague says "here, verify this match", is the verifier more motivated to concur with the initial analyst and as a result, rationalizes the decision (subconsciously) by "seeing more corresponding features" (than if that analyst had received the impression without the prompt).

Why don't we look then at the available studies in fingerprints to address the issue.

The current Dror et al studies show a bias effect, but they do not show us anything about how the effect was achieved. ANd in fact in one of their papers admit that low repeatability among examiner conclusions in difficult cases may be contributing to some of the bias effect (Dror 2007). I found the same thing--repeatability is an issue (Langenburg 2009). So we don't know and can't tell from Dror's studies-other than that a bias effect due to context information was demonstrated. ANd as pointed out in Langenburg, et al 2009, the bias effect that Dror et al have shown is solely towards producing inconclusive or exclusion decisions; they have not actually had a trial where an expert was prompted by contextual information into making an erroneous IDENTIFICATION decision, i.e. the Mayfield effect. So anecdotally, we know it exists, it just isn't evidenced in the Dror research...which says it may be more complicated than simple context bias effects. Nonetheless, this does not mean we should discount Dror's findings. The important point is they DID demonstrate a bias effect.

In Schiffer and Champod (2007) they showed that the analysis phase (feature selection-minutiae only) was fairly robust to bias effects. Telling the case type, or suspect confession etc. didn't necessarily show a measurable change in minutiae selection.

In Langenburg, et al in the J of Forensic Sci (2009) we saw something very relevant to this sports-magazine experiment. We had a control group comparing images and a low bias and a high bias group. In the bias groups, we did observe (like Dror) a bias effect on opinions reported (more inconclusives were reported in this group).

What is of key importance is that not only did they report more inconclusives in these groups, but also they reported less corresponding minutiae. I agree with some of the criticisms of our study that they "knew they were participating in an experiment (not a bias one though, but it's possible some caught on later)". However if that's the case, and bias studies can only be performed with the person unaware of participating in an experiment, then well, we need to discount a LOT of bias studies, including this sports article experiment, since obviously they were participating in an experiment.

So if we accept the value of our experiment here, and that experts in the bias group reported MORE INCONCLUSIVES...let's look at WHY. They reported observing less minutiae corresponding than the control group (and less variance too). [I have a forthcoming study in the thesis that shows the same effect in another experiment]. Selecting fewer or more minutiae from bias effects is unlikely to be something they would have been consciously aware of.

So either the experts were:
1) Reaching an inconclusive opinion (because of the bias prompt) and THEN RATIONALIZING it with (subconsciously) "seeing" fewer minutiae than the control group. OR
2) The bias effect directly affected the expert's abilities to see as many corresponding minutiae and since they saw less minutiae, reported Inconclusive.

It's a chicken or the egg thing. If #1 is true then yes, we have rationalization occuring. If #2 is true then we don't have any evidence for it yet.
I am on the fence with this one. The forthcoming research and the previous studies show that it may be a bit of both...and yet again more complex than realized.

Those are my thoughts on the issue. If anyone wants copies of the papers, email me. (they are on the most recent Grail disc if you have a copy of that).
I welcome some debate on this issue (or anecdotes if people care to share). I can think of cases/anecdotes that support both sides of this...just look at the McKie thread for amazing rationalization of decisions...

glenn.langenburg@state.mn.us

g.

[Pat A. Wertheim]

So either the experts were:
1) Reaching an inconclusive opinion (because of the bias prompt) and THEN RATIONALIZING it with (subconsciously) "seeing" fewer minutiae than the control group. OR
2) The bias effect directly affected the expert's abilities to see as many corresponding minutiae and since they saw less minutiae, reported Inconclusive.

To complicate things even more, might 1) [as listed above] be true for some participants, 2) [as listed above] be true for others,
and 3) [not listed above] deception/dishonesty be true for yet others?

And is it possible to design an experiment to separate 1), 2), and 3) ?

[g.]

To complicate things even more, might 1) [as listed above] be true for some participants, 2) [as listed above] be true for others,
and 3) [not listed above] deception/dishonesty be true for yet others?

And is it possible to design an experiment to separate 1), 2), and 3) ?

Just to be clear, you mean #3 being: someone perhaps realizes they have made an error and post hoc attempts to justify the position. It is more conscious and deliberate, as opposed to a subconscious bias effect. As has been alleged for example in some high profile cases...

And yeah, if that's what you're referring to, then it's outside my scope, since obviously in the experiments that have been conducted, I don't perceive that to be an issue.

Do I think it could be a either/or effect depending on the person. Possibly. As I said in the previous post, I am still sifting through some new data...but it certainly appears to be possible.

As for separating them out, yes I think it's possible in an experiment with sequential unmasking of information. To determine this in case work (or an experiment) it would be absolutely necessary to document the features observed first during an analysis and then document the features used in comparison to reach a decision. This is pretty similar to what the FBI is currently doing. They would be an excellent source of data on this issue in a case work environment, if they would (could) share.

Good question Pat.

g.

[Pat A. Wertheim]

Hi Glenn

Thinking about this more, I believe #3) I listed above (deception) might have been the underlying "rationalization" in the original experiment Lisa referred us to. In other words, the guy picks up the magazine with the blonde in the wet bikini, but when the researcher asks why he chose that one, he presents what he believes will sound like a rational reason (rationalizes) when in fact he knows it is not true. Isn't that a conscious lie? In that regard, "rationalization" could be a conscious lie or, in the case of your research, an unconsious, retrospective justification for a decision that was made, but not an outright, conscious lie.

I'm trying to relate the magazine experiment to your research, and it seems to be an apples and oranges thing. I'm going to have to cogitate on this a little more this morning using the famous Ron Smith Method -- "Drink more coffee!"

Pat

[David Fairhurst]

Did anyone else miss this the first time?

Participants examined the descriptions of the two magazines, circled the magazine
they would choose,

I don't think the magazines, and consequently the pictures of girls in swimsuits, were actually available to the men to pick up. The conclusion is therefore that simply the suggestion that the magazine has such pictures influences men.

[Dr. Dror]

This discussion touches upon many interesting and important issues; and I agree with many of the points the contributors have made to this thread. However, the issues are not simple; we have just finished studying a bunch of these issues, as they relate to forensics, investment decisions, and gambling. If you are interested in the details, I am happy to share an advanced draft summarizing the issues and studies (any comments are welcome, but please, as always, don't shoot the messenger!...
The draft manuscript is available at: http://cognitiveconsultantsinternationa ... Theory.doc

Thanks,


Itiel

[Gerald Clough]

Did anyone else miss this the first time?

Participants examined the descriptions of the two magazines, circled the magazine
they would choose,

I don't think the magazines, and consequently the pictures of girls in swimsuits, were actually available to the men to pick up. The conclusion is therefore that simply the suggestion that the magazine has such pictures influences men.

I thought about that, but I can decide if it makes any difference. They specified it was the Swimsuit Issue that was featured in whatever magazine they associated it with, and that issue is something their subjects were all familiar with. It would have been very difficult for them to concoct fabricated magazines, some with swimsuit issue covers and some without. But I'm not at all sure if they had made that effort it would have changed things in any way or changed the implications. Imagined images of the swimsuit issue are present in the subjects' minds, whether it's actually in front of them or just described as being in one magazine or another. Not having the actual images out there probably tends to put the various attributes on as equal footing as possible, similar to not showing actual illustrated and titled articles and other features and thus avoiding introducing more or less random preferences for stories about favorite teams or favored players or concocting fake articles about non-existent teams and artificially deflating their value to sports-savy subjects.

And something occurred to me later. I wonder if the predispositions to subscribe to a sports magazine at all is a factor. Suppose one subject cares little or nothing about sports, and another cares very much about sports. If told that they must subscribe to one of two offered sports magazines, the sports lover is likely to rate the informational attributes more highly than the could-care-less subject who will, if he must pick one, rate the swimsuit issue as having high value, since it's the only feature in which he could have an interest. And, if they didn't care that much about sports, they might well value the number of sports covered higher than the number articles, since that would likely include something the might be somewhat interested in, rather than a detailed article that was unlikely to attract them. Was the issue of predisposition ignored as a result in researchers prejudging the subjects, all college males, to have equal interests in sports? Harumph! Bias, I say, I say, son! Eeeeeeeevil bias!

Maybe they should run the study a couple of more times. Once with computer magazines (the PC World Swim Suit Issue?) with computer science majors and once with literary magazines (the Hot Librarians Issue?) with lit majors. Hah! Preselecting for geeks and nerds gives the swimsuit issue way too much power.