September 2008

Of Economics Both Rational and Irrational: With a Nod to Gilbert and Sullivan

For my military knowledge, though I'm plucky and adventury,
Has only been brought down to the beginning of the century;
But still, in matters vegetable, animal, and mineral,
I am the very model of a modern Major-General

I've had the opportunity over this summer to read a number of popular economics books (perhaps an oxymoron?) on the subject of rational and irrational economics. Within the rarified air of economics, a battle is going on among the adherents of behavioral economics, who agree that cognition and emotion affect decision-making, but apparently disagree whether these factors cause people to make decisions that are, in some final sense, rational or irrational.

The four books I read are:

Predictably Irrational: Dan Ariely
Sway: The Irresistible Pull of Irrational Behavior: Ori and Rom Brafman
The Undercover Economist: Tim Harford
The Logic of Life: The Rational Economics of an Irrational World: Tim Harford

All of these books are well-written and engaging, and rely, in great part, on explicating experiments that have been carried out by professional economists (indeed, the two books that focus on irrational decision-making are written by professional economists). The key issue being explored by these experiments is whether choices become less rational when emotion and beliefs are factored into the decision-making process.

The two books by Harford assert that decision-making is, at the end of the day, rational. You may have to dig to understand how it is rational, but dig enough, and rational you will find. By contrast, Ariely and the Brothers Brafman assert that decisions are swayed by emotional biases and emotional stimulation (i.e., arousal).

"The key issue ... is whether choices become less rational when emotion and beliefs are factored into the decision-making process."

Harford, speaking for the rational side, describes how the 1990s US “epidemic” of teenage oral sex was actually a widely adopted rational response to the increased risk of AIDS. Because sexual intercourse held the possibility of contracting this disease, oral sex was selected as a method of achieving satisfaction while lowering the possibility of getting AIDS. So, rationality triumphed. As an aside, it's easy to see that this debate can range far afield from the typical concerns of economics, which seem to focus on interest rates and profitability; certainly an improvement, from my perspective.

Ariely, treading on some of the same ground, weighs in for the irrational by describing an experiment he did that revolved around sex. To test decision-making under the state of arousal, they had Berkeley male undergraduates fill out a questionnaire about their sexual beliefs, willingness to engage in different sexual acts, etc. Then they took these undergraduates and put them into a state of arousal by having them view pornography, immediately after which they had them fill out the survey once more (the second survey conditions involved laptops wrapped with Saran-Wrap, raising the distasteful factor of this experiment considerably, though perhaps illustrating why interest in economics is up among undergraduate majors!).

The survey found that these virile young men significantly changed their answers to the same questions after becoming aroused, thereby demonstrating how decision-making is skewed (or tainted, perhaps) by arousal. By the way, the point of the experiment was to explore arousal in general, which would include anger, love, emotional attachment, and so on, not merely sexual stimulation. It was just that Ariely had concluded that presenting pornography to young men was an extremely reliable method of achieving arousal!

One might say that these are interesting aspects of this debate, but since the two situations are so dissimilar, they don't really address the debate.

Therefore, a more congruent set of experiments was designed.

This time, the irrational camp kicked of the debate by exploring irrational attachment and decision-making. A group of randomly chosen people were given cheap beer mugs decorated with beer manufacturer logos. In other words, people were given those cheesy glass mugs plastered with Budweiser labels. People were then asked to value these mugs, assessing the worth of other peoples' mugs as well as their own. Typically, they valued others' mugs at, say, fifty cents, while they assigned a much higher value to their own mugs – on the order of $1.75. Since the mugs were all the same, and were all brand-new, it's irrational to value a mug one has possessed for less than five minutes at three times the value of an identical mug owned by someone else for the same length of time.

The rational camp objected to this experiment on the grounds of its artificiality. I mean, cheesy beer mugs in a classroom? Of course there are weird outcomes.

So a similar experiment was performed, this time with enamel lapel pins. People were given one of two Disney-themed pins. Once again, people indicated irrational preferences, forming over-strong attachments (i.e., assigning a very high price) to their recently obtained pin, even if, they admitted, they would have preferred the other pin. In other words, their unreasonable commitment to “their” pin overrode their real desires – definitely an indication of irrationality.

However, the experimenters noted that the outcomes were skewed by the fact that the participants were naïve about pins and didn't have enough context to fairly value the pins. Their theory went that, if more experienced pin dealers participated in this same experiment, they would assign more rational prices to the pins, with a value appropriate for the pin and a willingness to pay a premium (on top of the reasonable price) for a more-desired pin. In other words, in a fair marketplace, knowledgeable participants will adhere to a rational price-setting strategy and optimize for personal preference by making marginal adjustments – just as someone will pay something extra to live in Short Hills, New Jersey rather than Newark, New Jersey, but not 100 times as much for a nearly identical dwelling. So, well done Hanford.

My precis here doesn't do justice to the fascinating nature of this debate (for a direct debate between Harford and Ariely, see http://www.omnivoracious.com/HarfordandAriely.html. Highly recommended). The impact of emotions and biases on decision-making is captivating to consider. Moreover, since every person makes decisions or is affected by those making decisions each day of the year, understanding the true nature of decision-making is vital to help bring about successful objective achievement.

I was reminded of this debate during a recent interchange I had regarding open source. A speaker scheduled to present at a conference I am involved with pulled out at the last minute with an apologetic note stating that he would be unable to participate due to resigning from his job.

“The decision came as a surprise to me,” the note said. This is certainly an odd formulation, since someone resigning from a job is not typically surprised by his or her action. To me, at least, this sounded more like someone who had been unexpectedly forced from his job rather than someone moving on to a better opportunity.

In discussing this, another conference participant said “Open source is bad for your career. Using it is a career-limiting move and makes you a target for someone higher-up who hates open source. I'm not sure why so many people hate open source, but they do. Maybe they just react against the 'true believers' of open source and dismiss it because they don't like the zealots.”

I'm reminded of survey after survey of senior IT management, who, when queried about the reasons they don't like open source, respond that they have concerns about support. Fair enough. Obviously, support is an important element in a software selection process, and ensuring that high-quality support is available is critical for important components of an organization's IT infrastructure.

However, one would hope this response would be based on appropriate evidence. That the organization would have experimented with several open source packages and evaluated the quality of support available. Or perhaps would have gone the extra step and used a formal assessment methodology (dare I suggest my own Open Source Maturity Model, which provides extensive ways to assess open source support quality as part of an overall assessment?) to objectively measure the quality of support available for a particular product.

If organizations had done so, they undoubtedly would have found that, generally speaking, the quality of support available for open source products is very good. Community support usually offers quick and relevant help. For those organizations unwilling to rely on community support, there are a number of companies offering, nay desperate, to provide high-quality commercial support.

There are numerous accounts of senior IT management rejecting open source due to its support limitations (indeed, I've had CIOs dismiss open source offhandedly with the statement “I need a throat to choke”), but to my knowledge I've never heard an account of someone who attempted to use an open source product but found that the product could not successfully be used due to support limitations – and, of course, there are ways to address those shortcomings if desired, up to and including hiring the actual developers of the product, which has to be the acme of support quality.

So one can conclude that this disdain for open source support reflects an decision based on emotional bias, reflecting than reasoned assessment.

Here's the thing, though: IT is supposed to be the pin dealers of technology – experienced, knowledgeable, aware of how to assess the value of market offerings; in short, IT should be rational on the question of open source. That's its charter and responsibility. Instead, too many IT managers react reflexively on the subject, rejecting without a second thought. So in this part of the debate, I'm afraid I'll have to side with Ariely and the Brafmans: irrationality plays a powerful part in decision-making, overriding pure rationality.

A second question is what that irrationality might mean, economically speaking – and here I'm speaking of traditional economics, concerned not with arousal patterns and beer mug attachment, but rather the question of what the financial outcomes of refusing to countenance open source might be.

On this question, I feel Harford holds all the cards.

He has an extended discussion in The Undercover Economist about profits and how they are created, protected, and distributed. He notes that profits vary according to the scarcity of the good or service provided by the profit-seeking party. For example, he feels that coffee growers will never do very well financially, since there are so many of them, it's difficult for any one grower to achieve larger profits, since another grower will always be ready to offer beans at a lower price. By the way, he has a very entertaining aside about how the current craze for Fair Trade coffee results in coffee merchants gathering large profits, with little trickling down (so to speak) to the growers themselves.

On the other hand, if you offer your goods or services in a situation of scarcity, you can command a premium and realize extra-high profits. Returning to coffee merchants, he notes that coffee stands inside of train stations charge extra-high prices because of their convenience for travelers and thus realize supersized profits.

If you are jumping ahead of the story and seeing the enterprise software giants as occupying the role of coffee merchants in New York's Grand Central Station, good for you – except in this example, it seems that not only do the software giants own the coffee stand, but they're also the landlord, coffee grower, and everything else in the supply chain. And just in case it wasn't scarce enough for them, they've been busily buying up their competitors to create even greater scarcity.

Harford, turning to another subject (trains), recounts a story about how he made a bet with an economist about how well the stock of the Great Eastern Railway (UK) did over its life. Harford, knowing how much trains transformed everyday society, imagined that GER must have just coined it, returning manyfold returns to its investors. Instead, he found that GER was a terrible investment, losing money for its investors during its lifespan.

This seems a dilemma: how can a railroad stock do so poorly, when trains changed life forever. Harford estimates that trains added 10 – 15% to Britain's GDP over the second half of the 19th century – an enormous effect – railroad investments, however, were a lousy investment.

Harford's explanation is that trains, while offering tremendous benefits to the society, suffered from significant competition and over-investment in the sector, causing poor financial results. In fact, you can see the result of this over-investment in London today, where two large stations – Kings Cross and St. Pancras – sit across the street from one another, a legacy of being constructed by two railroad lines that covered (roughly speaking) the same territory. Consequently, railroad investors lost their shirts, while UK inhabitants were able to wear a far wider choice of shirts, all made possible by trains.

With the rise of open source, shouldn't software be moving from the returns due to a well-placed coffee stand to those more like a coffee grower? There is no longer any reason for software scarcity, given the lack of license fees and freedom to redistribute that are part of open source licenses.

Certainly, from a purely financial economic point of view, open source represents an opportunity for organizations to realize the benefits of software application – moving the profits of enterprise software and much more to the income statements of software users.

But here we must leave the lands of the rational and return to “real life,” where irrationality prevails. By insisting on sticking with the tried-and-true solutions, and rejecting new providers, IT managers are hindering their employers from moving to the commodity side of the software equation – the side where the financial returns accrue to users, not vendors. More importantly, by failing to grapple with these new developments in their field, managers are failing in fulfilling their charter – to apply technology for the benefit of the firm. Instead, they're allowing sloth and prejudice to guide their decisions – at great cost to their employers. More open-minded competitors and industry newcomers, less locked into the previous mindset, are grasping the opportunity of open source today and realizing its financial opportunities.

Gilbert and Sullivan's Modern Major General is an officer up-to-date on every social trend, every technology development, every new event – except in his own field, where his knowledge has developed no further than the “beginning of the century,” i.e., 90 years before the present day. It doesn't take much imagination to recognize the effect that the Major General's lack of knowledge and willingness to learn would have on his troops and his country.

Just so impact of a senior IT manager content to deny open source's utility and reject its use. By adhering to conventional wisdom and following comfortable practices, he or she withholds the benefits of new software approaches from the overall firm. This consigns the company itself to an unpalatable situation: living with artificial software scarcity while allowing more nimble competitors to move to a more realistic software commodity world. In the short-term, using open source may be a career-limiting move, but in the long term, experiencing a revolution and failing to respond to it is a sure career-killer.

Navica News

Speaking Engagements

September 3, 2008, 6:00 p.m.: "Open Source Governance", Oakland, CA., East Bay Innovation Group Best Practices SIG. See here for more information

October 13, 2008:"Open Source in the Enteprise", Linux Foundation End User Summit, Desmond Tutu Conference Center, New York, New York. I will be discussing the "Open Source in the Enterprise" Report Findings

If you are interested in having me speak at your organization, contact me directly via email.