Author name cluster

Rosalind Picard

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

6 papers

1 author row

AAAI Conference 2020 Conference Paper

Hierarchical Reinforcement Learning for Open-Domain Dialog

Abdelrhman Saleh
Natasha Jaques
Asma Ghandeharioun
Judy Shen
Rosalind Picard

Open-domain dialog generation is a challenging problem; maximum likelihood training can lead to repetitive outputs, models have difﬁculty tracking long-term conversational goals, and training on standard movie or online datasets may lead to the generation of inappropriate, biased, or offensive text. Reinforcement Learning (RL) is a powerful framework that could potentially address these issues, for example by allowing a dialog model to optimize for reducing toxicity and repetitiveness. However, previous approaches which apply RL to open-domain dialog generation do so at the word level, making it difﬁcult for the model to learn proper credit assignment for long-term conversational rewards. In this paper, we propose a novel approach to hierarchical reinforcement learning (HRL), VHRL, which uses policy gradients to tune the utterance-level embedding of a variational sequence model. This hierarchical approach provides greater ﬂexibility for learning long-term, conversational rewards. We use selfplay and RL to optimize for a set of human-centered conversation metrics, and show that our approach provides signiﬁcant improvements – in terms of both human evaluation and automatic metrics – over state-of-the-art dialog models, including Transformers.

PDF Details

NeurIPS Conference 2019 Conference Paper

Approximating Interactive Human Evaluation with Self-Play for Open-Domain Dialog Systems

Asma Ghandeharioun
Judy Hanwen Shen
Natasha Jaques
Craig Ferguson
Noah Jones
Agata Lapedriza
Rosalind Picard

Building an open-domain conversational agent is a challenging problem. Current evaluation methods, mostly post-hoc judgments of static conversation, do not capture conversation quality in a realistic interactive context. In this paper, we investigate interactive human evaluation and provide evidence for its necessity; we then introduce a novel, model-agnostic, and dataset-agnostic method to approximate it. In particular, we propose a self-play scenario where the dialog system talks to itself and we calculate a combination of proxies such as sentiment and semantic coherence on the conversation trajectory. We show that this metric is capable of capturing the human-rated quality of a dialog model better than any automated metric known to-date, achieving a significant Pearson correlation (r>. 7, p<. 05). To investigate the strengths of this novel metric and interactive evaluation in comparison to state-of-the-art metrics and human evaluation of static conversations, we perform extended experiments with a set of models, including several that make novel improvements to recent hierarchical dialog generation architectures through sentiment and semantic knowledge distillation on the utterance level. Finally, we open-source the interactive evaluation platform we built and the dataset we collected to allow researchers to efficiently deploy and evaluate dialog models.

PDF Details

IJCAI Conference 2015 Conference Paper

Common Sense Reasoning for Detection, Prevention, and Mitigation of Cyberbullying (Extended Abstract)

Karthik Dinakar
Rosalind Picard
Henry Lieberman

We present an approach for cyberbullying detection based on state-of-the-art text classification and a common sense knowledge base, which permits recognition over a broad spectrum of topics in everyday life. We analyze a more narrow range of particular subject matter associated with bullying and construct BullySpace, a common sense knowledge base that encodes particular knowledge about bullying situations. We then perform joint reasoning with common sense knowledge about a wide range of everyday life topics. We analyze messages using our novel AnalogySpace common sense reasoning technique. We also take into account social network analysis and other factors. We evaluate the model on real-world instances that have been reported by users on Formspring, a social networking website that is popular with teenagers. On the intervention side, we explore a set of reflective userinteraction paradigms with the goal of promoting empathy among social network participants. We propose an air traffic control-like dashboard, which alerts moderators to large-scale outbreaks that appear to be escalating or spreading and helps them prioritize the current deluge of user complaints. For potential victims, we provide educational material that informs them about how to cope with the situation, and connects them with emotional support from others. A user evaluation shows that incontext, targeted, and dynamic help during cyberbullying situations fosters end-user reflection that promotes better coping strategies.

PDF Details

JBHI Journal 2015 Journal Article

Guest Editorial Sensor Informatics and Quantified Self

Rosalind Picard
Gary Wolf

The articles in this special issue focus on new technologies and applications for medical services that incorporate wearable sensors, signal processing, machine learning, and data mining techniques. The ability to collect large sets of human data comfortably 24/7, are advancing new ways to learn about human well being. Measurements that used to be confined to short-term sampling in a lab or medical facility are now able to be conducted continuously, while at home, work, sleep, or play. Studies are no longer limited to a focus on disease progression or to the effect of therapeutic measures provided in clinical settings—instead, it is becoming possible to quantify healthy activities and behavior, and capture how these slowly change as illness develops or progresses. The quantified self movement, where people can monitor their own health and fitness-related data, is closely linked to the emergence of new methods in biometric sensing.

Details DOI

AAAI Conference 2014 Conference Paper

Modeling Subjective Experience-Based Learning under Uncertainty and Frames

Hyung-il Ahn
Rosalind Picard

In this paper we computationally examine how subjective experience may help or harm the decision maker’s learning under uncertain outcomes, frames and their interactions. To model subjective experience, we propose the “experienced-utility function” based on a prospect theory (PT)-based parameterized subjective value function. Our analysis and simulations of two-armed bandit tasks present that the task domain (underlying outcome distributions) and framing (reference point selection) influence experienced utilities and in turn, the “subjective discriminability” of choices under uncertainty. Experiments demonstrate that subjective discriminability improves on objective discriminability by the use of the experienced-utility function with appropriate framing for a given task domain, and that bigger subjective discriminability leads to more optimal decisions in learning under uncertainty.

PDF Details

NeurIPS Conference 2005 Conference Paper

Hyperparameter and Kernel Learning for Graph Based Semi-Supervised Classification

Ashish Kapoor
Hyungil Ahn
Yuan Qi
Rosalind Picard

There have been many graph-based approaches for semi-supervised clas- siﬁcation. One problem is that of hyperparameter learning: performance depends greatly on the hyperparameters of the similarity graph, trans- formation of the graph Laplacian and the noise model. We present a Bayesian framework for learning hyperparameters for graph-based semi- supervised classiﬁcation. Given some labeled data, which can contain inaccurate labels, we pose the semi-supervised classiﬁcation as an in- ference problem over the unknown labels. Expectation Propagation is used for approximate inference and the mean of the posterior is used for classiﬁcation. The hyperparameters are learned using EM for evidence maximization. We also show that the posterior mean can be written in terms of the kernel matrix, providing a Bayesian classiﬁer to classify new points. Tests on synthetic and real datasets show cases where there are signiﬁcant improvements in performance over the existing approaches.

PDF Details