🇺🇸Crowdsourcing Bayesian Networks with Prediction Markets

Charles Twardy, Ph.D., KeyW Corporation & George Mason University

Presented at the 6th Annual BayesiaLab Conference in Chicago, November 1-2, 2018.

Abstract

In this talk, we look at how the SciCast system uses prediction markets to efficiently crowdsource both the structure and parameters of a Bayesian network. Prediction markets are among the most accurate methods to combine forecasts; forecasters form a consensus probability distribution by trading contingent securities. A combinatorial prediction market forms a consensus joint distribution over many related events by allowing conditional trades or trades on Boolean combinations of events. In general, that is infeasible, but using a Bayesian network underneath makes it tractable for large problems. As a bonus, the market solves some difficulties with Bayes net elicitation, including knowledge bottlenecks, divided or overlapping expertise, and some impossibility theorems that affect other aggregation methods. We have a formal paper in press in J. Artificial Intelligence Research.

Presentation Video

Presentation Slides

About the Presenter

Dr. Charles R. Twardy is a Senior Data Scientist with KeyW Corporation and an affiliate professor at George Mason University. From 2011 to 2015, he led the George Mason DAGGRE and SciCast teams for the IARPA ACE and ForeST forecasting challenges. The final result was a novel Bayesian prediction market that allowed users to make forecasts conditional on the outcomes of other forecasts at scale on the fly. Forecast accuracy improved by about 35% over baseline, with much greater expressivity.

From 2015 to 2016, as a Senior Data Scientist for NTVI Federal, he helped the Defense Suicide Prevention Office establish procedures for handling and analyzing sensitive personal data and provided sound approaches to hot-spot detection. In late 2016, he joined Sotera (now KeyW), where he worked on two DARPA big data programs and on IARPA CREATE, a successor to IARPA ACE.

Dr. Twardy focuses on evidence and inference, with a special interest in causal models, Bayesian networks, Bayesian search theory (especially wilderness search), and information theory. He has also worked on argument mapping, trajectory clustering, sensor-selection, counter-IED models, source credibility, image recognition, environmental decision-making, and epidemiology.

He received a dual Ph.D. in Cognitive Science and History & Philosophy of Science from Indiana University and a B.A. from the Interdisciplinary Majors program at the University of Virginia.