Nicholas Bartlett

NeurIPS Conference 2011 Conference Paper

Hierarchically Supervised Latent Dirichlet Allocation

• Adler Perotte
• Frank Wood
• Noemie Elhadad
• Nicholas Bartlett

We introduce hierarchically supervised latent Dirichlet allocation (HSLDA), a model for hierarchically and multiply labeled bag-of-word data. Examples of such data include web pages and their placement in directories, product descriptions and associated categories from product hierarchies, and free-text clinical records and their assigned diagnosis codes. Out-of-sample label prediction is the primary goal of this work, but improved lower-dimensional representations of the bag-of-word data are also of interest. We demonstrate HSLDA on large-scale data from clinical document labeling and retail product categorization tasks. We show that leveraging the structure from hierarchical labels improves out-of-sample label prediction substantially when compared to models that do not.

ICML Conference 2010 Conference Paper

Forgetting Counts: Constant Memory Inference for a Dependent Hierarchical Pitman-Yor Process

• Nicholas Bartlett
• David Pfau
• Frank Wood

NeurIPS Conference 2010 Conference Paper

Probabilistic Deterministic Infinite Automata

• David Pfau
• Nicholas Bartlett
• Frank Wood

We propose a novel Bayesian nonparametric approach to learning with probabilistic deterministic finite automata (PDFA). We define and develop and sampler for a PDFA with an infinite number of states which we call the probabilistic deterministic infinite automata (PDIA). Posterior predictive inference in this model, given a finite training sequence, can be interpreted as averaging over multiple PDFAs of varying structure, where each PDFA is biased towards having few states. We suggest that our method for averaging over PDFAs is a novel approach to predictive distribution smoothing. We test PDIA inference both on PDFA structure learning and on both natural language and DNA data prediction tasks. The results suggest that the PDIA presents an attractive compromise between the computational cost of hidden Markov models and the storage requirements of hierarchically smoothed Markov models.