Author name cluster

Matthew Le

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

3 papers

1 author row

NeurIPS Conference 2023 Conference Paper

Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale

Matthew Le
Apoorv Vyas
Bowen Shi
Brian Karrer
Leda Sari
Rashel Moritz
Mary Williamson
Vimal Manohar

Large-scale generative models such as GPT and DALL-E have revolutionized the research community. These models not only generate high fidelity outputs, but are also generalists which can solve tasks not explicitly taught. In contrast, speech generative models are still primitive in terms of scale and task generalization. In this paper, we present Voicebox, the most versatile text-guided generative model for speech at scale. Voicebox is a non-autoregressive flow-matching model trained to infill speech, given audio context and text, trained on over 50K hours of speech that are not filtered or enhanced. Similar to GPT, Voicebox can perform many different tasks through in-context learning, but is more flexible as it can also condition on future context. Voicebox can be used for mono or cross-lingual zero-shot text-to-speech synthesis, noise removal, content editing, style conversion, and diverse sample generation. In particular, Voicebox outperforms the state-of-the-art zero-shot TTS model VALL-E on both intelligibility (5. 9\% vs 1. 9\% word error rates) and audio similarity (0. 580 vs 0. 681) while being up to 20 times faster. Audio samples can be found in \url{https: //voicebox. metademolab. com}.

PDF Details

AAAI Conference 2014 Conference Paper

Spatio-Temporal Consistency as a Means to Identify Unlabeled Objects in a Continuous Data Field

James Faghmous
Hung Nguyen
Matthew Le
Vipin Kumar

Mesoscale ocean eddies are a critical component of the Earth System as they dominate the ocean’s kinetic energy and impact the global distribution of oceanic heat, salinity, momentum, and nutrients. Therefore, accurately representing these dynamic features is critical for our planet’s sustainability. The majority of methods that identify eddies from satellite observations analyze the data in a frame-by-frame basis despite the fact that eddies are dynamic objects that propagate across space and time. We introduce the notion of spatio-temporal consistency to identify eddies in a continuous spatiotemporal field, to simultaneously ensure that the features detected are both spatially and temporally consistent. Our spatio-temporal consistency approach allows us to remove most of the expert criteria used in traditional methods to reduce false negatives. The removal of arbitrary heuristics enables us to render more complete eddy dynamics by identifying smaller and longer lived eddies compared to existing methods.

PDF Details

AAAI Conference 2013 Conference Paper

Multiple Hypothesis Object Tracking For Unsupervised Self-Learning: An Ocean Eddy Tracking Application

James Faghmous
Muhammed Uluyol
Luke Styles
Matthew Le
Varun Mithal
Shyam Boriah
Vipin Kumar

Mesoscale ocean eddies transport heat, salt, energy, and nutrients across oceans. As a result, accurately identifying and tracking such phenomena are crucial for understanding ocean dynamics and marine ecosystem sustainability. Traditionally, ocean eddies are monitored through two phases: identiﬁcation and tracking. A major challenge for such an approach is that the tracking phase is dependent on the performance of the identiﬁcation scheme, which can be susceptible to noise and sampling errors. In this paper, we focus on tracking, and introduce the concept of multiple hypothesis assignment (MHA), which extends traditional multiple hypothesis tracking for cases where the features tracked are noisy or uncertain. Under this scheme, features are assigned to multiple potential tracks, and the ﬁnal assignment is deferred until more data are available to make a relatively unambiguous decision. Unlike the most widely used methods in the eddy tracking literature, MHA uses contextual spatio-temporal information to take corrective measures autonomously on the detection step a posteriori and performs signiﬁcantly better in the presence of noise. This study is also the ﬁrst to empirically analyze the relative robustness of eddy tracking algorithms.

PDF Details