Danny Wyatt

TIST Journal 2011 Journal Article

Inferring colocation and conversation networks from privacy-sensitive audio with implications for computational social science

• Danny Wyatt
• Tanzeem Choudhury
• Jeff Bilmes
• James A. Kitts

New technologies have made it possible to collect information about social networks as they are acted and observed in the wild, instead of as they are reported in retrospective surveys. These technologies offer opportunities to address many new research questions: How can meaningful information about social interaction be extracted from automatically recorded raw data on human behavior? What can we learn about social networks from such fine-grained behavioral data? And how can all of this be done while protecting privacy? With the goal of addressing these questions, this article presents new methods for inferring colocation and conversation networks from privacy-sensitive audio. These methods are applied in a study of face-to-face interactions among 24 students in a graduate school cohort during an academic year. The resulting analysis shows that networks derived from colocation and conversation inferences are quite different. This distinction can inform future research in computational social science, especially work that only measures colocation or employs colocation data as a proxy for conversation networks.

AAAI Conference 2010 Conference Paper

Discovering Long Range Properties of Social Networks with Multi-Valued Time-Inhomogeneous Models

• Danny Wyatt
• Tanzeem Choudhury
• Jeff Bilmes

The current methods used to mine and analyze temporal social network data make two assumptions: all edges have the same strength, and all parameters are time-homogeneous. We show that those assumptions may not hold for social networks and propose an alternative model with two novel aspects: (1) the modeling of edges as multi-valued variables that can change in intensity, and (2) the use of a curved exponential family framework to capture time-inhomogeneous properties while retaining a parsimonious and interpretable model. We show that our model outperforms traditional models on two real-world social network data sets.

AAAI Conference 2008 Conference Paper

Learning Hidden Curved Exponential Family Models to Infer Face-to-Face Interaction Networks from Situated Speech Data

• Danny Wyatt

In this paper, we present a novel probabilistic framework for recovering global, latent social network structure from local, noisy observations. We extend curved exponential random graph models to include two types of variables: hidden variables that capture the structure of the network and observational variables that capture the behavior between actors in the network. We develop a novel combination of informative and intuitive conversational (local) and structural (global) features to specify our model. The model learns, in an unsupervised manner, the relationship between observable behavior and hidden social structure while simultaneously learning properties of the latent structure itself. We present empirical results on both synthetic data and a real world dataset of face-to-face conversations collected from 24 individuals using wearable sensors over the course of 6 months.

IJCAI Conference 2007 Conference Paper

• Danny Wyatt
• Tanzeem Choudhury
• Jeff Bilmes
• Henry Kautz

In this paper we introduce a new dynamic Bayesian network that separates the speakers and their speaking turns in a multi-person conversation. We protect the speakers' privacy by using only features from which intelligible speech cannot be reconstructed. The model we present combines data from multiple audio streams, segments the streams into speech and silence, separates the different speakers, and detects when other nearby individuals who are not wearing microphones are speaking. No pre-trained speaker specific models are used, so the system can be easily applied in new and different environments. We show promising results in two very different datasets that vary in background noise, microphone placement and quality, and conversational dynamics.

AAAI Conference 2005 Conference Paper

Unsupervised Activity Recognition Using Automatically Mined Common Sense

• Danny Wyatt

A fundamental difficulty in recognizing human activities is obtaining the labeled data needed to learn models of those activities. Given emerging sensor technology, however, it is possible to view activity data as a stream of natural language terms. Activity models are then mappings from such terms to activity names, and may be extracted from text corpora such as the web. We show that models so extracted are sufficient to automatically produce labeled segmentations of activity data with an accuracy of 42% over 26 activities, well above the 3. 8% baseline. The segmentation so obtained is sufficient to bootstrap learning, with accuracy of learned models increasing to 52%. To our knowledge, this is the first human activity inferencing system shown to learn from sensed activity data with no human intervention per activity learned, even for labeling.