Tag Archives: Forecasting

Can Game Theory Predict Conflict?

New York University’s Bueno de Mesquita presented his research a TED earlier this year and the New York Times Magazine recently featured his work on conflict forecasting. His forthcoming book, “The Predictioneer’s Game,” is written for a popular audience and includes dozens of stories the forecasts he has made over the past 20 years.

Professor de Mesquita has developed a computer model that allegedly predicts the outcome of any situation in which political parties try to persuade or coerce one another.

“Since the early 1980s, C.I.A. officials have hired him to perform more than a thousand predictions; a study by the C.I.A., now declassified, found that Bueno de Mesquita’s predictions ‘hit the bull’s-eye’ twice as often as its own analysts did.”

That’s not saying much if the CIA’s accuracy was miniscule to begin with. In any case, vaccording to de Mesquita’s calculations, Iran won’t be making a nuclear bomb. His forecast suggests that Iran will get very close to developing one, but will then cease from going any further.

If his predictions are correct, then all the “dire portents we’ve seen in recent months—the brutal crackdown on protesters, the dubious confessions, Khamenei’s accusations of American subterfuge—are masking a tectonic shift. The moderates are winning, even if we cannot see that yet.”

Critics of de Mesquita’s work argue that “the proprietary software he uses can’t be trusted and may cast doubt on the larger enterprise of making predictions.” I tend to be sceptical about people in the business of making predictions. What I find surprising about de Mesquita’s predictions on Iran is the apparent lack of reference to the recent violence. Why didn’t his model correctly predicted the election fiasco and political wrestling that ensued?

Advertisements

Applying Earthquake Physics to Conflict Analysis

I’ve long found the analogies between earthquakes and conflicts intriguing. We often talk of geopolitical fault lines, mounting tensions and social stress. “If this sounds at all like the processes at work in the Earth’s crust, where stresses build up slowly to be released in sudden earthquakes … it may be no coincidence” (Buchanan 2001).

To be sure, violent conflict is “often like an earthquake: it’s caused by the slow accumulation of deep and largely unseen pressures beneath the surface of our day-to-day affairs. At some point these pressures release their accumulated energy with catastrophic effect, creating shock waves that pulverize our habitual and often rigid ways of doing things…” (Homer-Dixon 2006).

But are fore shocks and aftershocks in social systems really as discernible as well? Like earthquakes, both inter-state and internal wars actually occur with the same statistical pattern (see my previous blog post on this). Since earthquakes and conflicts are complex systems, they also exhibit emergent features associated with critical states. In sum, “the science of earthquakes […] can help us understand sharp and sudden changes in types of complex systems that aren’t geological–including societies…” (Homer-Dixon 2006).

The Model

To this end, I collaborated with Professor Didier Sornette and Dr. Ryan Woodard from the Swiss Federal Institute of Technology (ETHZ) to assess whether a mathematical technique developed for earthquake prediction might shed light on conflict dynamics. I presented this study along with our findings at the American Political Science Association (APSA) convention last year (PDF).

This geophysics technique, “superposed epoch analysis,” is used to identify statistical signatures before and after earthquakes. In other words, this technique allows us to discern whether any patterns are discernible in the data during foreshocks and aftershocks.

Earthquake physicists work from global spatial time series data of seismic events to develop models for earthquake prediction. We used a global time series dataset of conflict events generated from newswires over a 15-year period. The graph below explains the “superposed epoch analysis” technique as applied to conflict data.

eqphysics

The curve above represents a time series of conflict events (frequency) over a particular period of time. We select arbitrary threshold, such as “threshold A” denoted by the dotted line. Every peak that crosses this threshold is then “copied” and “pasted” into a new graph. That is, the peak, together with the data points 25 days prior to and following the peak is selected.

The peaks in the new graph are then superimposed and aligned such that the peaks overlap precisely. With “threshold A”, two events cross the threshold, five for “threshold B”. We then vary the thresholds to look for consistent behavior and examine the statistical behavior of the 25 days before and after the “extreme” conflict event.

Results

For this study, we performed the computational technique described above on the conflict data for the US, UK, Afghanistan, Columbia and Iraq.

Picture 4Picture 5Picture 6

The foreshock and aftershock behaviors in Iraq and Afghanistan appear to be similar. Is this because the conflicts in both countries were the result of external intervention, i.e., invasion by US forces (exogenous shock)?

In the case of Colombia, an internal low intensity and protracted conflict, the statistical behavior of foreshocks and aftershocks are visibly different from those of Iraq and Afghanistan. Do the different statistical behaviors point to specific signature associated with exogenous and endogenous causes of extreme events? Does one set of behavior contrast with another one in the same way that old wars and new wars differ?

Future Research

Are certain extreme events endogenous or exogenous in nature? Can endogenous or exogenous signatures be identified? In other words, are extreme events just part of the fat tail of a power law due to self-organized criticality (endogeneity)? Or is catastrophism in action, extreme events require extreme causes outside the system (exogeneity)?

Another possibility still is that extreme events are the product of both endogenous and exogenous effects. How would this dynamic unfold? To answer these questions, we need to go beyond political science.

The distinction between responses to endogenous and exogenous processes is a fundamental property of physics and is quantified as the fluctuation-dissipation theorem in statistical mechanics. This theory has been successfully applied to social systems (such as books sales) as a way to help understand different classes of causes and effects.

Our goal is to use the same techniques to investigate the questions: Do conflict among actors in social systems display measurable endogenous and exogenous behavior?  If so, can a quantitative signature of precursory (endogenous) behavior be used to help recognize and then reduce growing conflict? The next phase of this research will be to apply the above techniques to the conflict dataset already used to examine the statistical behavior of foreshocks and aftershocks.

Mirror, Mirror on the Wall…

Which conflict forecasting model is the most accurate of them all? None are accurate to begin with. Has no one read Nassim Taleb’s “The Black Swan“?

Taleb

Recent empirical studies demonstrate that experts, i.e., us (and our sophisticated systems and methodologies) are only marginally better than novices in our ability to accurately forecast political and economic events. Furthermore, these studies show that neither group’s forecasts are much better than random guessing.

Of greater concern still is the empirical observation that experts nevertheless remain consistently overconfident of the accuracy of their own forecasts. This is compared to novices who tend to be more conservative vis-à-vis their forecasting abilities although they are equally (in)effective when it comes to accuracy. A separate study found that “somehow, the analysts’ self-evaluation did not decrease their error margin after their failures to forecast.”

Perhaps the most telling test of how academic methods fare in the real world was run by Spyros Makridakis, “who spent part of his career managing competitions between forecasters who practice a ‘scientific method’ called econometrics […]. Simply put, he made people forecast in real life and then he judged their accuracy” (1).

This led to the following lamentable conclusion “statistically sophisticated or complex methods do not provide more accurate forecasts than simpler ones” (2). And so, despite the fact that “billions of dollars have been invested in developing sophisticated data banks and early warnings, we have to note that even the most expensive systems have shown a striking inability to forecast political events,” not to mention galvanize any preventive measures (Rupesinghe 1988).

What I really don’t understand, however, is how some experts profess to forecast conflicts and at the same time use the word “discontinuous” to describe trends in conflict. If a process experiences tipping points (or punctuated equilibria) then no econometric model however fancy can provide accurate forecasts. Talk to anyone at the Santa Fe Institute (SFI) if you’re not convinced. Or read this piece by Charles Doran on “Why Forecasts Fail.”

100 Year Early Warning from 1900?

In December 1900, The Ladies’ Home Journal published a fascinating article by Elfreth Watkins on “What May Happen in the Next 100 Years” (PDF of original article here). Elfreth opens with the words “These prophecies will seem strange, almost impossible. Yet they will have come from the most learned and conservative minds in America.”

Here are some of the 29 prophesies that ensued:

  • There will be No C, X or Q in our every-day alphabet. They will be abandoned because unnecessary. Spelling by sound will have been adopted.
  • There will be no street cars within our large cities. All hurry traffic will be below or high above the ground.
  • Photographs will be telegraphed from any distance. If there is a battle in China in a hundred years hence, snapshots of its most striking events will be published in the newspapers an hour later.
  • There will be airships, but they will not successful compete with surface cars and water vessels but they will be maintained as deadly war vessels by all military nations.
  • There will be no wild animals except in menageries. Rats and mice will have been exterminated. The horse will have become practically extinct.
  • Man will see around the world. Persons and things of all kinds will be brought within focus of cameras connected electrically with screens at opposite ends of circuits, thousands of miles at a span. American audiences in their theatres will view upon huge curtains before them the coronations of kings in Europe or the progress of battles in the Orient. The instrument bringing these distant scenes to the very doors of people will be connected with a giant telephone apparatus transmitting each incidental sound in its appropriate place. Thus the guns of a distant battle will be heard to boom when seen to blaze, and thus the lips of a remote actor or singer will be heard to utter words or music when seen to move.
  • Wireless telephone and telegraph circuits will span the world. A husband in the middle of the Atlantic will be able to converse with his wife sitting in her boudoir in Chicago. We will be able to telephone to China quite as readily as we now talk from New York to Brooklyn. By an automatic signal they will connect with any circuit in their locality without the intervention of a “hello girl”.
  • A university education will be free to every man and woman. Several great national universities will have been established.
  • Pneumatic tubes, instead of store wagons, will deliver packages and bundles. These tubes will collect, deliver and transport mail over certain distances, perhaps for hundreds of miles. They will at first connect with the private houses of the wealthy; then with all homes. Great business establishments will extend them to stations, similar to our branch post-offices of today, whence fast automobile vehicles will distribute purchases from house to house.

Hindsight is, of course, always 20/20. But note that the lack of any reference to global wars and global institutions despite the note in passing about “air ships” being used for warfare.

Improvements in Conflict Forecasting? (Updated Post)

I exchanged several emails with Professor Jack Goldstone today regarding an article of his (co-authored with seven other professors) currently under review: “A Global Model for Forecasting Political Instability.” I got in touch with Jack after a reference in David Nyheim‘s recent report on early warning caught my eye. Namely that conflict forecasting tools were now producing 80% accurate forecasts.

Jack did confirm the reference. Most of the Political Instability Task Force (PITF) models now have an 80%+ accuracy rate vis-a-vis forecasting state crises 2 years in advance. “There is a global model which averages 81.6% accuracy across 3 random control sets for both event and non-event forecasting, and a sub-Saharan Africa model that averages about 85%.” The article is still under review, so for now the piece should be referenced as “Working Paper, Center for Global Policy, George Mason University.”

Jack kindly shared the abstract with me:

Applying case-control methodology to data from countries worldwide from 1955 to 2003, the authors develop a statistical model of political instability that distinguishes with roughly 80-percent accuracy between countries headed for new crises two years hence and those that will remain stable. The resulting model employs few of the variables championed by those who write on political instability and is comparatively simple, using only a few variables, all but one of them categorical. A measure of regime type emerges as the most powerful predictor of instability in the two-year time frame, leading the authors to conclude that political institutions, not economic conditions, demography, or geography, are the most important predictors of the onset of political instability.

I find it particularly interesting that regime type appears to be the most salient predictor of political instability. What are the implications for conflict early response and humanitarian intervention? When I asked Jack why he and his colleagues had not applied their model through to 2007/2008 in order to make predictions for 2008-2010, he mentioned that the organization sponsoring the research has exclusive rights to the data and predictions about the present and near future. This is unfortunate, but of course I understand that this is beyond the team’s control. But it does beg the following question: does closed, proprietary research on conflict forecasting models contribute to the warning-response gap?

Update: Professor Goldstone kindly shared additional information. While he and his research team cannot use the material, their sponsors are making extensive use of the forecasts. Indeed, Centcom is drawing on the conflict modeling and parts of the forecasts are also being incorporated into the OECD fragile states response planning. So while the funder wishes to control dispersal, they are not keeping it fully locked up.

It would be useful if we could identify which operational responses/interventions (either by Centcom or the OECD) can be traced back to decision-making processes that were directly influenced by information generated from PITF’s models.