Fast and Frugal Early Warning

Bradley Perry, a colleague of mine who just completed his Masters of Science in Applied Intelligence, kindly shared a copy of his excellent MA thesis entitled “Fast and Frugal Conflict Early Warning in Sub-Saharan Africa: The Role of Intelligence Analysis.” Bradley carried out his study to counter the erroneous assumption that only those who have large budgets and operate outside high-conflict regions (e.g., academics) are best placed to engage in conflict early warning analysis.

Instead of drawing on dozens and dozens of indicators like the majority of early warning systems, which necessitates substantial amounts of data (most of which is highly aggregated and/or of poor quality), Bradley takes just three indicators to forecast conflict escalation: income inequality, ethnic fractionalization and political freedom. The results from this “good enough” model suggest that we should question our field’s inclination for data-intensive methodologies. In Bradley’s own words, “the results do argue that both the conflict early warning and intelligence communities should consider the value of fast and frugal analysis.”

The fact that the conflict early warning field has been riddled with data-intensive methodologies for the past 20 years is directly due to the fact that those designing these methodologies are for the most part hardcore academics obsessed with prediction and sophisticated econometric models. To be sure, “most conflict early warning systems rely on resource intensive methods. They often take years to develop, and are built on complicated algorithms that require vast amounts of data.” As Bradley adds, however:

“It is safe to say no one has created a system that has the ability to predict; it is likely that no one ever will. In fact, if prediction were the goal in conflict early warning, the intelligence field would have little to offer. Former US government intelligence analyst and author of Anticipating Surprise: Analysis for Strategic Warning, Cynthia Grabo gives this caveat: ‘Warning is not a fact, a tangible substance, a certainty, or a provable hypothesis. It is not something which the finest collection system should be expected to produce full blown or something which can be delivered to the policymaker with the statement, ‘Here it is. We have it now.’”

Below are excerpts from Bradley’s research that strongly resonated with my experience in the field of conflict early warning:

William G. Nhara, a former advocate for the establishment of an early warning system for the OAU, suggests that an early warning system for the African context should be based on a number of methodologies; rather than detail how their incorporation into one system might appear however, he merely lists general sources of information to include: historical surveys and analyses of events, analyses of the content of documents and reports, comparative analyses of relevant information, physical inspections and field visits, statistical sampling and inference, operations research techniques, economic and econometric analysis, and modeling and remote sensing. This enumeration offers little explanation as to how the analyst might process the information, except to say that the responsible agency should store it in a database.

Robert Mudida, a professor of International Conflict Management at the University of Nairobi, described the AU’s “Situation Room” as merely one set up with CNN TV. According to him, it is not proving to be an effective institution in regards to prediction (R. Mudida, pers. comm.).

Indigenous organizations, those with the most responsibility and the greatest chance for success in conflict early warning, are spending precious, yet scant, resources in research, development, and implementation of these models. However, if it is accurately feasible to avoid the complex set of indicators that accompany most warning models and skip altogether the danger of having otherwise accurate systems fall short in applicability, then the identification of a “good enough” model is worth pursuing.

Advertisements

5 responses to “Fast and Frugal Early Warning

  1. One of the things that impressed me most about this thesis is the deep understanding that Bradley shows concerning Africa. Resources for this kind of thing ARE limited. The simple but generally effective model makes sense.

    Kris

  2. Thanks for your comments, Kris, I wholeheartedly agree.

  3. Lawrence Woocher

    Patrick,

    I’m very interested in the idea of “fast and frugal” early warning and am grateful for your passing along Bradley’s interesting thesis. I have a few reactions.

    First, there are surely some data-intensive early warning efforts around, notably in Africa. But how widespread is the assumption that effective early warning requires these resource- and data-intensive models? To the contrary, I’d say that most policymakers (and probably most people in the field at large) discount the utility of statistical models for early warning. Gareth Evans’ comments on this are representative: “While a great deal of work has been done on early warning signs, particularly in the case of conflict generally, with some attempt to build complex quantitative models as well as lists of qualitative indicators, this enterprise remains for the most part an art rather than a science” (p. 74 of his new book, The Responsibility to Protect). He goes on to identify five factors he believes one should consider in making “these essentially seat of the pants judgments.” Bradley writes, “Most conflict early warning systems rely on resource intensive methods.” Yet most conflict early warning is not “systematized” but rather informal, relying primarily on qualitative analysis by country experts.

    Second, the most rigorous statistical work done to develop a forecasting model has generated surprisingly simple results. I refer to the Political Instability Task Force and its global forecasting model of instability (see http://globalpolicy.gmu.edu/pitf/PITFglobal.pdf). Their effort has surely been resource- and data-intensive, and has included testing lots of complex models, but the results are neither particularly data-intensive nor complex. One could easily mistake the PITF membership as fitting Patrick’s description of “hardcore academics obsessed with prediction and sophisticated econometric models.” This makes their relatively simple results all the more noteworthy.

    Third, depending on one’s measure of “good enough,” it may be possible to develop even faster and more frugal forecasting models. I recall a researcher telling me that using two variables (infant mortality and population size), one can get upwards of 70+% accuracy at forecasting episodes of mass killing. Many people have suggested that a few smart people in a room could forecast at least as accurately as any statistical model.

    I’ve written previously that a risk assessment/early warning strategy should balance three attributes: (1) accuracy in estimating risk, (2) efficiency/feasibility of using the methods in question, and (3) perceived legitimacy of the process by key stakeholders. In the end, the best approach will depend on the context. So I fully support efforts to develop models that perform as accurately as extant ones, but with greater efficiency. Likewise, there is value in striving to improve the accuracy of our models, even if they become somewhat less efficient.

    Lastly, I don’t think the biases of academics engaged as consultants are sufficient to explain data-heavy and resource intensive early warning systems in Africa—a region that one assumes should focus on efficiency. The politics of governments in the region are at least as important, I believe. For example, I’d hazard to guess that the ECOWARN system has 90+ indicators not because any academic would recommend so many, but because it was easier to forge political consensus by adding more and more indicators, instead of winnowing the list down to a few powerful ones.

    Thanks again, Patrick.

    Lawrence

  4. I appreciate Lawrence’s feedback on Bradley’s thesis on Early Warning.

    As a final year Master student of Conflict Analysis and Peacebuilding, I find that ”Fast and Fugal” conflict early warning has a lot of substantial information as to how an efficient, yet simple, early warning system should be developed, using Sub-Saharan Africa as a test tube.

    Resources on this field are indeed limited.

    In India, Meta-Culture, an organisation dedicated to dialogue and conflict transformation is in the process of developing a conflict early warning system in Bangalore, and Bradley’s thesis has been a valuable source in the development of this system.

    Thank you, Bradley.
    Stacy-Ann

    • Hi Stacy-Ann,

      Many thanks for your comment, I completely agree with you and also very much commend Bradley the arguments he makes in his thesis. Where could we get more information on your work in India?

      Thanks again,
      Patrick

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s