APL Wins DARPA Contract to Uncover Hidden Ground Truth in Social Phenomena

The Defense Advanced Research Projects Agency (DARPA) has awarded the Johns Hopkins University Applied Physics Laboratory (APL) in Laurel, Maryland, a performer role in the Ground Truth Program to test whether state-of-the-art social science modeling methods can uncover hidden “ground truth” in simulated but complex social systems.

Over the course of the up to 2.5-year, $3.7-million effort, APL, with its collaborators at JHU Whiting School of Engineering, Cornell University, University of Hawaii, and Graphika, Inc., will attempt to reverse engineer artificial worlds, unpacking how and why simulated agents acted the way they did and predicting future outcomes. The ultimate goal is to help DARPA improve its knowledge of the capabilities and limitations of an array of social science modeling methods.

The Department of Defense has long turned to social scientists to better understand the social, cultural, behavioral and political forces that shape regions of the world with strategic importance to the United States. Current approaches to studying population trends rely on a variety of methods — both qualitative and quantitative — that make inferences about the causes of social phenomena based on observations in the real world.

But in the real world, the “ground truth” is rarely known, making it impossible to know how accurate these methods and models really are, or whether the connections they observe and predict are matters of cause and effect or just correlations. This can lead to spurious conclusions about how elements of the system affect each other and support less than optimal decisions.

In the Ground Truth program, a group of research teams will build multiple social simulations based on rules known only to them. Other teams — including one led by APL — will be challenged to come up with a set of approaches for discovering those rules. Then, DARPA and an independent test and evaluation team will score the modeling teams’ abilities to identify and predict causal ground truth in the series of simulations with different degrees of social complexity.

If successful, the agency says, Ground Truth will demonstrate a principled approach for testing the power and limitations of various social science modeling methods, explore new modeling approaches for describing and predicting different kinds of complex social systems, and inform future investments for understanding the complex interplay of social, cultural, behavioral and political factors.

Finding out the true causal pathways and mechanisms makes a big difference in creating and executing interventions to those outcomes, says Ground Truth Principal Investigator Aurora Schmidt, of APL’s Research and Exploratory Development Department.

“Much of research in general, including social science, relies on looking for correlations among factors,” she says. “But just because, for example, being male makes one more likely to major in computer science doesn’t mean that males necessarily have more aptitude for computer science. If we want to design interventions to get more females to consider STEM fields, then it helps to be able to pinpoint the causative factors (rather than correlative ones) that result in lower participation.”

Currently there are a wide array of tools and methods aimed at figuring out causal relationships, but the community is unsure how much confidence to place in the different approaches to explaining complex social systems. This program provides a way to validate the tools for social science questions and can reveal which methods are most suited to which questions and situations. If successful, this work would enable researchers to uncover the causal factors that matter most.

APL’s work with Ground Truth builds upon the recently completed collaboration “Tracking Critical-Mass Outbreaks in Social Contagions,” led by Cornell University and in collaboration with Graphika, Inc. In this three-year effort, funded by the Department of Defense Minerva Initiative, researchers studied emerging hashtags and topical communications in social media to test theoretical models of social contagion and then used this real-world data to determine if social contagions associated with mass mobilization events can be detected.

“We compared the explanatory abilities of complex versus simple contagion models against real-world adoption cascades and found evidence that we can detect political and controversial topics simply by the dynamics of the cascade,” explained Schmidt.

“Our Ground Truth team will quantitatively test the explanatory and predictive power of social science theories on large-scale population datasets where influence processes affect the behavior of individual actors,” she explained. “The program is constructed to use synthetically generated populations for study, allowing the team to test a variety of methods for discovering causal models as well as determine when there is insufficient information to answer certain types of social science questions about the mechanisms underlying the observed phenomena.”