Mirror, Mirror on the Wall…

Which conflict forecasting model is the most accurate of them all? None are accurate to begin with. Has no one read Nassim Taleb’s “The Black Swan“?

Taleb

Recent empirical studies demonstrate that experts, i.e., us (and our sophisticated systems and methodologies) are only marginally better than novices in our ability to accurately forecast political and economic events. Furthermore, these studies show that neither group’s forecasts are much better than random guessing.

Of greater concern still is the empirical observation that experts nevertheless remain consistently overconfident of the accuracy of their own forecasts. This is compared to novices who tend to be more conservative vis-à-vis their forecasting abilities although they are equally (in)effective when it comes to accuracy. A separate study found that “somehow, the analysts’ self-evaluation did not decrease their error margin after their failures to forecast.”

Perhaps the most telling test of how academic methods fare in the real world was run by Spyros Makridakis, “who spent part of his career managing competitions between forecasters who practice a ‘scientific method’ called econometrics […]. Simply put, he made people forecast in real life and then he judged their accuracy” (1).

This led to the following lamentable conclusion “statistically sophisticated or complex methods do not provide more accurate forecasts than simpler ones” (2). And so, despite the fact that “billions of dollars have been invested in developing sophisticated data banks and early warnings, we have to note that even the most expensive systems have shown a striking inability to forecast political events,” not to mention galvanize any preventive measures (Rupesinghe 1988).

What I really don’t understand, however, is how some experts profess to forecast conflicts and at the same time use the word “discontinuous” to describe trends in conflict. If a process experiences tipping points (or punctuated equilibria) then no econometric model however fancy can provide accurate forecasts. Talk to anyone at the Santa Fe Institute (SFI) if you’re not convinced. Or read this piece by Charles Doran on “Why Forecasts Fail.”

About these ads

2 responses to “Mirror, Mirror on the Wall…

  1. Pingback: New Media, Accuracy and Balance of Power in Crises « iRevolution

  2. The book mentioned above, written by Nassim Taleb is a good strong read on this subject. Worthy of anyone in the conflict management or study field, taking time to give it a read.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s