Wednesday, August 4, 2010
Many years ago, as a budding prestidigitator, I made a promise that I would never lie to the audience. Thus, I used clever phrasing to lead them to conclusions of their own making. Later as an IT professional (magic it seems doesn’t pay all that well), I found that fulfilling user requirements was almost as difficult as divining the secrets of Houdini, Blackstone, Copperfield, and Paul Gertner. Maybe it was just me, but it seems the more information I gave my users, the more they changed their minds.
How we decide is the title of a great book by Jonah Lehrer which covers a lot of ground, but has one particularly interesting chapter that all IT decision-makers should read. It begins with a simple question, “Which brand of jam do you prefer?”
In the 1980’s, Consumer Reports brought together a group of sensory experts along with the leading brands of jam. They then ranked the jams by sixteen different criteria. A few years later the University of Virginia replicated the test with undergradute students and came pretty much to the same results. So much for sensory experts.
But when the University asked the participants “Why” they chose as they did, the participants started to introduce all kinds of new variables into the equation, and (and this is the key point) they changed their minds. Jams that once rated high, were rated poor and vice versa. In Lehrer’s book, he makes the point (based on significant research) that given lots of extraneous data our rational minds tend to invent reasons to use the additional data, often looking for patterns that simply don’t exist.
Consider the research performed at Stanford and Caltech involving wines priced from $5.00 to $45.00 per bottle. The more expensive wines were consistently selected as being better when, in some cases, the wines were from the same bottle; just priced as if they were different. These studies of how we make choices have been conducted under rigorous conditions including MRIs to provide clear evidence of how our brains are making each decision.
In short - given more data (such as the price of a glass of wine) our brains will attempt to use that data to “improve” our decisions - - even when the additional data has no bearing on the quality of the product, choice, or outcome.
If you are tasked with selecting a vendor product from a group, limit the number of variables to the elements that truly matter (no small task), and avoid looking at more data - even as a tie breaker. If the core data elements cannot distinguish one selection over another; go with your gut - don’t add more data.