flaw in Black Swan theory: few events are "unexpected"
I've been pondering Nassim Nicholas Taleb's "Black Swan" theory lately, and am wondering if maybe he missed a critical point. Perhaps it's not so much that consequential events result from unforeseen happenings, but rather that momentum prevents people from taking actions until it is too late.
Take the credit bubble as an example. Not only was it predictable that there would be a fall-out from the loose underwriting standards (e.g. giving mortgages to people with poor credit, no income, etc), but we have numerous people on record for having foreseen this credit crunch years ago. Apocalyptic books were written on the subject early in the millennium, and there were even insiders who fought against the trends (the Wall Street Journal has documented how some conservative investment bankers were shown the door because they fought against the growing bets on CDOs and dodgy mortgages).
Aside from the current credit crisis, even the .com crash was predicted years before it occurred, with none other than Alan Greenspan stating that stocks were rising beyond any rational basis (hence the famous "irrational exuberance" speech).
It's perfectly understandable that investment bankers would marginalize anyone who suggested they cut back on mortgage securitization deals that were minting billions of dollars in profits (and padding their bonuses) back in 2005. So what if there might be some storm clouds out on the horizon, if you are making a fortune at something today? No one is going to walk away from the feed trough prematurely, and will instead stay until there is nothing left. Why would a successful software firm completely change its business model (that is currently highly successful), and products, when there might still be many years of gravy still ahead? Who wants to leave all that money on the table?
Ironically, it is often the people who's intuitions are correct who end up suffering the most. Sure, you might be right in predicting some major future event, but bucking the trends (and social pressures) has consequences. As I mentioned earlier, more than a handful of investment bankers lost their jobs in recent years due to their reluctance to "get on board" with the easy credit movement. In the end, many people who followed the trend might still come out ahead. So what if a mortgage broker gets fired today? They were making $400,000 a year a short while ago. It will take someone else making $70,000 many years to equal the same earning capacity.
My point is this: game-changing events are almost always foreseeable, but social forces prevent most organizations from doing anything about it.
The implications are clear. It is unlikely that any business will be undone from some new technology, or trend, emerging from out of the blue. Rather, the disruptive events that occur won't be surprises at all (except to those who willfully closed their eyes), and in retrospect we will find that organizations marginalized, or expunged, those people who "saw" the future and tried to do something about it.
Unfortunately, the trendiness of Messr Taleb's Black Swan theory is almost becoming a convenient excuse for companies who run into trouble. Hey, massively significant unpredictable events happen all the time, so we can't be responsible for getting caught with our pants down. How were we to know that the once in a 500 year flood was going to happen now?
Take the credit bubble as an example. Not only was it predictable that there would be a fall-out from the loose underwriting standards (e.g. giving mortgages to people with poor credit, no income, etc), but we have numerous people on record for having foreseen this credit crunch years ago. Apocalyptic books were written on the subject early in the millennium, and there were even insiders who fought against the trends (the Wall Street Journal has documented how some conservative investment bankers were shown the door because they fought against the growing bets on CDOs and dodgy mortgages).
Aside from the current credit crisis, even the .com crash was predicted years before it occurred, with none other than Alan Greenspan stating that stocks were rising beyond any rational basis (hence the famous "irrational exuberance" speech).
It's perfectly understandable that investment bankers would marginalize anyone who suggested they cut back on mortgage securitization deals that were minting billions of dollars in profits (and padding their bonuses) back in 2005. So what if there might be some storm clouds out on the horizon, if you are making a fortune at something today? No one is going to walk away from the feed trough prematurely, and will instead stay until there is nothing left. Why would a successful software firm completely change its business model (that is currently highly successful), and products, when there might still be many years of gravy still ahead? Who wants to leave all that money on the table?
Ironically, it is often the people who's intuitions are correct who end up suffering the most. Sure, you might be right in predicting some major future event, but bucking the trends (and social pressures) has consequences. As I mentioned earlier, more than a handful of investment bankers lost their jobs in recent years due to their reluctance to "get on board" with the easy credit movement. In the end, many people who followed the trend might still come out ahead. So what if a mortgage broker gets fired today? They were making $400,000 a year a short while ago. It will take someone else making $70,000 many years to equal the same earning capacity.
My point is this: game-changing events are almost always foreseeable, but social forces prevent most organizations from doing anything about it.
The implications are clear. It is unlikely that any business will be undone from some new technology, or trend, emerging from out of the blue. Rather, the disruptive events that occur won't be surprises at all (except to those who willfully closed their eyes), and in retrospect we will find that organizations marginalized, or expunged, those people who "saw" the future and tried to do something about it.
Unfortunately, the trendiness of Messr Taleb's Black Swan theory is almost becoming a convenient excuse for companies who run into trouble. Hey, massively significant unpredictable events happen all the time, so we can't be responsible for getting caught with our pants down. How were we to know that the once in a 500 year flood was going to happen now?
Comments
I think it's a flaw however to even compare any bubble to a black swan. Essentially every bubble shows signs of popping prior to it's eventual demise. Smart people always announce the disconnect, then it performs one final upward push...and pop.
A real black swan is more like company X invests $$$ in technology Y, only to discover that once their investment is made new technology Z is patented which is half the cost and twice as good. Such an event really is a black swan. But perhaps the point is that something truly unexpected can probably only happen on a micro scale, rather than on a large scale.
That awareness needs to be compelling enough to affect the public perception of the steps needed. A crackdown on mortgage lending might not have been appreciated. Following 9-11, there was concern about scattered evidence of a general threat was ignored that should have led to stricter airport screening. Now there is better tracking of those threats, along with evidence of various probing of defenses going on in flights, and yet still there is complaining about the steps taken in response.
I am not so sure that even the examples of revolutionary technological developments hold up as "black swans". Automobiles had been around for 20 years or so before they really started to threaten horses and buggies. It took decades from the first invention of the light-bulb before the electrical infrastructure became wide-spread enough for them to be a viable light source for most people.
In almost every case where some new invention disrupted the existing economic interests the upstart had been in the wings for a good while before it gained critical mass, and could really upset the status quo.
In fact, there are numerous examples where existing companies acknowledged the growing threats, but failed to do anything substantive about it.
I don't think this lack of action (to deal with new developing trends) by established firms indicates that they are run by imbeciles. Rather, I suspect it is simply difficult to ever justify throwing out an existing practice that is still making money even if storm clouds can be seen somewhere off on the horizon. Heck, look how many people are reluctant to abandon their homes and property even when it is clear a hurricane is bearing down on them within the next 12 hours!
I personally worked at a mainframe computer manufacturer in the early '90s. The personal computing trend had existing for over a decade, yet we had our biggest year ever for mainframe sales in 1990. Sure, there were people at our company who were worried about Unix systems, and PCs. Our firm even toyed around with loading Unix on mainframes, and adding PC networking type functionality.
In the end, however, we went down in flames in VERY short order. My conclusion was that it was simply impossible for a company that was constructed around a mainframe business model to re-invent itself as something else altogether. Were PCs and Unix workstations a "black swan" for my company? Hardly! Although many of the people at my firm were in utter shock at how quickly the end came when it did. They had become used to thumbing their noses at everyone who was saying their business was doomed for nearly a decade. Everyone kept telling us that "mainframes were dead", but we still kept minting amazing profits.
Maybe it is this exact inflection point, where the new technology FINALLY supplants the old stalwarts that is the "black swan". But that is a very different kind of thing than just a surprise technological development the likes of which Taleb speaks.
.....Perhaps it's not so much that consequential events result from unforeseen happenings, but rather that momentum prevents people from taking actions until it is too late.....
Really interesting post sniglet. I would suggest that a distinction of a good manager is that action would be taken before it is too late, despite conventional wisdom.
The first sentence In the Foreword (written by Bernard M. Baruch) of John MacKay's Extraordinary Popular Delusions and the Madness of Crowds is:
All economic movements, by their very nature, are motivated by crowd psychology....
In case you have not heard of it, Extraodinary Delusions was written in 1852 and includes the famous Tulipmania story.
.
I think the system is constructed precisely to ensure that "good" managers aren't allowed. As I mentioned earlier, any investment banking CEO who was unwilling to gorge on the credit bubble would have been summarily fired by shareholders during the boom years. How could you explain why your bank's profits (and revenues) were SO much lower than that of its peers? There are known cases where conservative investment bankers were shoved out the door to prevent them getting in the way of the bubble.
My example of the mainframe manufacturer that continued to defy the trends for many years is very instructive. Was it really "wrong" to continue making money selling mainframes even though many people recognized it couldn't go on forever?
Maybe the really hard part is timing. Sure, the change in trend might be obvious, but who wants to leave good money on the table prematurely? Unfortunately, it's almost never possible to time the changes perfectly, and hop off at exactly the right moment. By the time the mainframe business went up in smoke it was too late to re-orient the company. But if the company had made a real attempt to change earlier maybe they would have lost billions in potential mainframe sales (which still occured for a number of years). It's hard (and likely impossible) to truly hedge your bets, adopting the new trend while still playing out the old one at the same time.
sniglet said -
...There are known cases where conservative investment bankers were shoved out the door to prevent them getting in the way of the bubble....
That may be true, but I bet those conservative investment bankers are not being investigated by the FBI right now either.
I would expect a good manager to recognize a bubble for what it is, after all, there is literature on the subject that goes back over 150 years. I would also expect a good manager to have the inegrity and courage to go against popular opinion as necessary. A manager who does not act on his own convictions is not a leader, but yet another sheep.
Also, why didn't the mainframe manufacturer, that you use in your example, diversify and transition into other product lines if they knew market demand was changing? Maybe that company's real problem was a lack of flexibility? You make it sound like it was an either / or situation where cautious transition was not possible.
.
The company (where I used to work) did see that things were changing and that's precisely why they ported Unix to their mainframes, as well as PC networking/file-sharing capabilities.
Unfortunately, the core issue was that the company was focussed around sophisticated engineering of high-end hardware that could be sold for high prices. We had no expertise in cheap mass-production, so going head to head with a Dell or Compaq was out of the question. Neither were we a "software" company. Our manufacturing plants were not designed to produce cheap hardware.
In other words, we needed to be a different company altogether to be a success. I suppose we could have started hiring people for new divisions focussed on completely new things, but the idea of somehow using the same production lines, engineers, or staff, that were making mainframes for producing Unix workstations or software just wasn't feasible.
Successful businesses become very good, and specialized, at doing something well. It is difficult to start doing something else completely different at the same time. It's one thing for a farmer to decide to diversify his crops because off changing markets and quite another to have an auto mechanic start working as a nurse.
Good discussion and thanx for explaining.
Another example may be how the Swiss mechanical watch industry was nearly wiped out when the digital / quartz watch was developed. Now that situation had to be a "black swan" event, yes?
Actually, the "Quartz Crisis" you cite is an excellent example of my main point that Black Swans don't exist. Quartz technology was actually pioneered in Switzerland! Swiss watchmakers could have had a leg up on everyone (i.e. since they had the technology first), but chose to stick with their existing mechanical designs.
Initially Quartz watches were only successful on the very low end, and it took years to move up the market.
http://en.wikipedia.org/wiki/Quartz_crisis
This is a PERFECT illustration of why established organizations will refuse to adopt new technologies until they are against the wall. The whole economics and business model of Quartz watches was completely different than the traditional mechanical ones. Instead of highly skilled artisans, quartz watches were mass produced with low-skilled labourers in factories. I can well understand how the talented Swiss craftsmen were reluctant to just become unemployed. Their skills were absolutely irrelevant to the new technology. And even when it became apparent that quartz watches were becoming a trend, why would a Swiss watchmaker want to just quit altogether when the higher-end products were still going to sell well for a good 5 more years or so?
Like I said before, it's hard for any company to run completely different businesses. It's often easier to just start all over with a new company than attempt to transition an old one.
By the way, many of my friends at the mainframe company re-trained and made new successful careers in other industries.
sniglet -
I reread this thread and if you are saying that the real estate / credit bubble would not qualify as a black swan event, I would agree to an extent.
I have not read any of Nicholas Taleb's writings so maybe I don't understand the full meaning of the black swan theory. Wikipedia defines it as a large impact, hard to predict, and rare event and the Sept 11, 2001 attacks are used as an example.
I think factors that you may not be considering are awareness and perception of probability. RCC and jon touched on these points. It has been reported that the 911 attacks were predicted as a possibility, but the small number of people who were even aware of the possibility and could have done something about it apparently considered the event improbable. (There are other opinions including those that claim that the current administration is stupid, but I won't get into that.)
The real estate / credit bubble would be a black swan event to people who believe that real estate always appreciates.
The demise of the mainframe would be a black swan event to those who believe that mainframes would always have a niche in the marketplace.
The impact of the digital / quartz watch on the Swiss watch-making industry would be a black swan event to those who believed that a watch is a finely crafted peice of jewelry instead of just a tool from which you can tell time.
If everyone had perfect awareness, I would agree that black swan events are fallacy.
Well, every event is a surprise to someone. So maybe the definition of a "black swan" is all about the perspective of the individual. You might have been the only one not to realize the oncoming train was going to hit your car, but if you were still surprised, then the train was a black swan.
I do think there is a qualitative difference between surprise events such as the 9/11 attacks, which very few people knew were going to happen although many knew that terrorists were "up to something" and something like the Quartz Crisis. Quartz technology, and even the emergence of cheap mass produced quartz time-pieces were not a surprise to ANYONE. What was a surprise to Swiss watch makers was how appealing these products eventually became.
So what is the black swan? The quartz watches or the market acceptance of them?
By the way, this all raises an interesting thought: the people who come up with great new ideas and technologies are rarely the ones to profit from it. The Swiss invented Quartz time-keeping technology but it was Japan that made a mint off of it. Xerox invented revolutionary new graphical computer interfaces but it was Apple that made successful products from it. It's enough to make one wonder if having great ideas is really as important to financial success as many people believe?
It seems like this can only be argued from a perspective of scale. The social perspective and common knowledge is that parking on active railroad tracks has extremely high risk. Societies tend to form consensus opinion to varying degrees. Perhaps, a black swan is any event where societal consensus is very strong and very one sided, but nonetheless proves to be false.
Thus, nationwide declining home prices in an expanding economy is very much a black swan event, since the major consensus was that it was impossible outside of some depression event. Of course, this definition only side steps the problem, because now we must decide what groups of people we consider to be "societies" for the black swan hypothesis.
sniglet said:
Well, every event is a surprise to someone. So maybe the definition of a "black swan" is all about the perspective of the individual. You might have been the only one not to realize the oncoming train was going to hit your car, but if you were still surprised, then the train was a black swan...
.
If you go by the Wikipedia definition of "Black Swan", a person being hit by a train would not qualify as a black swan event because it would not have a "large impact" (Unless you are the person being hit. It's hard to imagine an impact larger than being hit by a train
I would totally agree that a black swan event is dependent on the perspective / awareness of the individual. Were the 911 attacks a black swan event to the analysts who predicted them?
..
Maybe it's not the train itself that is the black swan, but the impact...
The Swiss watchmakers knew all about quartz time-keeping technology, and even the fact that the Japanese were starting to use it. But they didn't percieve that this would have a negative impact on their own businesses. Many investment bankers knew we were in a credit bubble (Chuck Prince at Citigroup said so in early 2007), but they didn't realize how swift, or deadly, the bubble collapse would be.
Hey, there are people who knowingly decide to sit out hurricanes (even throwing parties), blithely stating that they've seen hurricanes before and that they are no big deal. How were they to know that this particular hurricane was going to be more devastating than the last two they had lived through?