Jason Weston

ICLR Conference 2025 Conference Paper

Backtracking Improves Generation Safety

• Yiming Zhang
• Jianfeng Chi
• Hailey Nguyen
• Kartikeya Upasani
• Daniel M. Bikel
• Jason Weston
• Eric Michael Smith

Text generation has a fundamental limitation almost by definition: there is no taking back tokens that have been generated, even when they are clearly problematic. In the context of language model safety, when a partial unsafe generation is produced, language models by their nature tend to happily keep on generating similarly unsafe additional text. This is in fact how safety alignment of frontier models gets circumvented in the wild, despite great efforts in improving their safety. Deviating from the paradigm of approaching safety alignment as prevention (decreasing the probability of harmful responses), we propose backtracking, a technique that allows language models to "undo" and recover from their own unsafe generation through the introduction of a special [RESET] token. Our method can be incorporated into either SFT or DPO training to optimize helpfulness and harmlessness. We show that models trained to backtrack are consistently safer than baseline models: backtracking Llama-3-8B is four times more safe than the baseline model (6.1\% $\to$ 1.5\%) in our evaluations without regression in helpfulness. Our method additionally provides protection against four adversarial attacks including an adaptive attack, despite not being trained to do so.

ICML Conference 2025 Conference Paper

Learning to Plan & Reason for Evaluation with Thinking-LLM-as-a-Judge

• Swarnadeep Saha
• Xian Li 0003
• Marjan Ghazvininejad
• Jason Weston
• Tianlu Wang

LLM-as-a-Judge models generate chain-of-thought (CoT) sequences intended to capture the step-by-step reasoning process that underlies the final evaluation of a response. However, due to the lack of human-annotated CoTs for evaluation, the required components and structure of effective reasoning traces remain understudied. Consequently, previous approaches often (1) constrain reasoning traces to hand-designed components, such as a list of criteria, reference answers, or verification questions and (2) structure them such that planning is intertwined with the reasoning for evaluation. In this work, we propose EvalPlanner, a preference optimization algorithm for Thinking-LLM-as-a-Judge that first generates an unconstrained evaluation plan, followed by its execution, and then the final judgment. In a self-training loop, EvalPlanner iteratively optimizes over synthetically constructed evaluation plans and executions, leading to better final verdicts. Our method achieves a new state-of-the-art performance for generative reward models on RewardBench and PPE, despite being trained on fewer amount of, and synthetically generated, preference pairs. Additional experiments on other benchmarks like RM-Bench, JudgeBench, and FollowBenchEval further highlight the utility of both planning and reasoning for building robust LLM-as-a-Judge reasoning models.

NeurIPS Conference 2025 Conference Paper

Meta CLIP 2: A Worldwide Scaling Recipe

• Yung-Sung Chuang
• Yang Li
• Dong Wang
• Ching-Feng Yeh
• Kehan Lyu
• Ramya Raghavendra
• Jim Glass
• Lifei Huang

Contrastive Language-Image Pretraining (CLIP) is a popular foundation model, supporting from zero-shot classification, retrieval to encoders for multimodal large language models (MLLMs). Although CLIP is successfully trained on billion-scale image-text pairs from the English world, scaling CLIP's training further to learning from the worldwide web data is still challenging: (1) no curation method is available to handle data points from non-English world; (2) the English performance from existing multilingual CLIP is worse than its English-only counterpart, i. e. , "curse of multilinguality" that is common in LLMs. Here, we present Meta CLIP 2, the first recipe training CLIP from scratch on worldwide web-scale image-text pairs. To generalize our findings, we conduct rigorous ablations with minimal changes that are necessary to address the above challenges and present a recipe enabling mutual benefits from English and non-English world data. In zero-shot ImageNet classification, Meta CLIP 2 ViT-H/14 surpasses its English-only counterpart by 0. 8% and mSigLIP by 0. 7%, and surprisingly sets new state-of-the-art without system-level confounding factors (e. g. , translation, bespoke architecture changes) on multilingual benchmarks, such as CVQA with 57. 4%, Babel-ImageNet with 50. 2% and XM3600 with 64. 3% on image-to-text retrieval. Code and model are available at https: //github. com/facebookresearch/MetaCLIP.

NeurIPS Conference 2025 Conference Paper

NaturalReasoning: Reasoning in the Wild with 2.8M Challenging Questions

• Weizhe Yuan
• Jane Yu
• Song Jiang
• Karthik Padthe
• Yang Li
• Dong Wang
• Ilia Kulikov
• Kyunghyun Cho

Scaling reasoning capabilities beyond traditional domains such as math and coding is hindered by the lack of diverse and high-quality questions. To overcome this limitation, we introduce a scalable approach for generating diverse and challenging reasoning questions, accompanied by reference answers. We present NaturalReasoning, a comprehensive dataset comprising 2. 8 million questions that span multiple domains, including STEM fields (e. g. , Physics, Computer Science), Economics, Social Sciences, and more. We demonstrate the utility of the questions in NaturalReasoning through knowledge distillation experiments which show that NaturalReasoning can effectively elicit and transfer reasoning capabilities from a strong teacher model. Furthermore, we demonstrate that NaturalReasoning is also effective for unsupervised self-training using external reward models or self-rewarding.

ICML Conference 2025 Conference Paper

R. I. P. : Better Models by Survival of the Fittest Prompts

• Ping Yu
• Weizhe Yuan
• Olga Golovneva
• Tianhao Wu 0002
• Sainbayar Sukhbaatar
• Jason Weston
• Jing Xu 0014

Training data quality is one of the most important drivers of final model quality. In this work, we introduce a method for evaluating data integrity based on the assumption that low-quality input prompts result in high variance and low quality responses. This is achieved by measuring the rejected response quality and the reward gap between the chosen and rejected preference pair. Our method, Rejecting Instruction Preferences (RIP) can be used to filter prompts from existing training sets, or to make high quality synthetic datasets, yielding large performance gains across various benchmarks compared to unfiltered data. Using Llama 3. 1-8B-Instruct, RIP improves AlpacaEval2 LC Win Rate by 9. 4%, Arena-Hard by 8. 7%, and WildBench by 9. 9%. Using Llama 3. 3-70B-Instruct, RIP improves Arena-Hard from 67. 5 to 82. 9, from 18th place to 6th overall in the leaderboard.

NeurIPS Conference 2025 Conference Paper

Self-Challenging Language Model Agents

• Yifei Zhou
• Sergey Levine
• Jason Weston
• Xian Li
• Sainbayar Sukhbaatar

Large language models are quickly becoming the foundation for intelligent agents that are capable of using tools. However, training such agents is challenging because it requires human creation and annotation of a diverse set of tasks, tools, and evaluation criteria. In this paper, we propose the Self-Challenging Agent framework for training an agent on high-quality tasks that are generated by itself. The agent first plays the role of challenger and generates a task after interacting with the given tools. The tasks take the form of a novel general class of problems termed Code-as-Task, which are defined by an instruction, a verification function and solution and failure cases which serve as tests, allowing to filter only for high-quality tasks. The agent then takes an executor role and trains on those tasks with reinforcement learning using the evaluation feedback as a reward. We show our method improves the performance of Llama-3. 1-8B-Instruct on two existing multi-turn tool-use agent benchmarks, M$^3$ToolEval and TauBench, with a two-fold average success rate increase, despite using only self-generated training data.

ICML Conference 2025 Conference Paper

Self-Consistency Preference Optimization

• Archiki Prasad
• Weizhe Yuan
• Richard Yuanzhe Pang
• Jing Xu 0014
• Maryam Fazel-Zarandi
• Mohit Bansal
• Sainbayar Sukhbaatar
• Jason Weston

Self-alignment, whereby models learn to improve themselves without human annotation, is a rapidly growing research area. However, existing techniques often fail to improve complex reasoning tasks due to the difficulty of assigning correct rewards. An orthogonal approach that is known to improve correctness is self-consistency, a method applied at inference time based on multiple sampling in order to find the most consistent answer. In this work, we extend the self-consistency concept to help train models. We thus introduce self-consistency preference optimization (ScPO), which iteratively trains consistent answers to be preferred over inconsistent ones on unsupervised new problems. We show ScPO leads to large improvements over conventional reward model training on reasoning tasks such as GSM8K and MATH, closing the gap with supervised training with gold answers or preferences, and that combining ScPO with standard supervised learning improves results even further. On ZebraLogic, ScPO finetunes Llama-3 8B to be superior to Llama-3 70B, Gemma-2 27B, and Claude-3 Haiku.

ICML Conference 2025 Conference Paper

Thinking LLMs: General Instruction Following with Thought Generation

• Tianhao Wu 0002
• Janice Lan
• Weizhe Yuan
• Jiantao Jiao
• Jason Weston
• Sainbayar Sukhbaatar

LLMs are typically trained to answer user questions or follow instructions similarly to how human experts respond. However, in the standard alignment framework they lack the basic ability of explicit thinking before answering. Thinking is important for complex questions that require reasoning and planning – but can be applied to any task. We propose a training method for equipping existing LLMs with such thinking abilities for general instruction following without use of additional human data. We achieve this by an iterative search and optimization procedure that explores the space of possible thought generations, allowing the model to learn how to think without direct supervision. For each instruction, the thought candidates are scored using a judge model to evaluate their responses only, and then optimized via preference optimization. We show that this procedure leads to superior performance on AlpacaEval and Arena-Hard, and shows gains from thinking on non-reasoning categories such as marketing, health and general knowledge, in addition to more traditional reasoning & problem-solving tasks.

NeurIPS Conference 2024 Conference Paper

Iterative Reasoning Preference Optimization

• Richard Y. Pang
• Weizhe Yuan
• Kyunghyun Cho
• He He
• Sainbayar Sukhbaatar
• Jason Weston

Iterative preference optimization methods have recently been shown to perform well for general instruction tuning tasks, but typically make little improvement on reasoning tasks. In this work we develop an iterative approach that optimizes the preference between competing generated Chain-of-Thought (CoT) candidates by optimizing for winning vs. losing reasoning steps. We train using a modified DPO loss with an additional negative log-likelihood term, which we find to be crucial. We show reasoning improves across repeated iterations of this scheme. While only relying on examples in the training set, our approach results in increasing accuracy on GSM8K, MATH, and ARC-Challenge for Llama-2-70B-Chat, outperforming other Llama-2-based models not relying on additionally sourced datasets. For example, we see a large improvement from 55. 6% to 81. 6% on GSM8K and an accuracy of 88. 7% with majority voting out of 32 samples.

ICLR Conference 2024 Conference Paper

Self-Alignment with Instruction Backtranslation

• Xian Li 0003
• Ping Yu
• Chunting Zhou
• Timo Schick
• Omer Levy
• Luke Zettlemoyer
• Jason Weston
• Mike Lewis

We present a scalable method to build a high quality instruction following language model by automatically labelling human-written text with corresponding instructions. Our approach, named instruction backtranslation, starts with a language model finetuned on a small amount of seed data, and a given web corpus. The seed model is used to construct training examples by generating instruction prompts for web documents (self-augmentation), and then selecting high quality examples from among these candidates (self-curation). This data is then used to finetune a stronger model. Finetuning LLaMa on two iterations of our approach yields a model that outperforms all other LLaMa-based models on the Alpaca leaderboard not relying on distillation data, demonstrating highly effective self-alignment.

ICML Conference 2024 Conference Paper

Self-Rewarding Language Models

• Weizhe Yuan
• Richard Yuanzhe Pang
• Kyunghyun Cho
• Xian Li 0003
• Sainbayar Sukhbaatar
• Jing Xu 0014
• Jason Weston

We posit that to achieve superhuman agents, future models require superhuman feedback in order to provide an adequate training signal. Current approaches commonly train reward models from human preferences, which may then be bottlenecked by human performance level, and secondly these reward models require additional human preferences data to further improve. In this work, we study Self-Rewarding Language Models, where the language model itself is used via LLM-as-a-Judge prompting to provide its own rewards during training. We show that during Iterative DPO training, not only does instruction following ability improve, but also the ability to provide high-quality rewards to itself. Fine-tuning Llama 2 70B on three iterations of our approach yields a model that outperforms many existing systems on the AlpacaEval 2. 0 leaderboard, including Claude 2, Gemini Pro, and GPT-4 0613. While there is much left still to explore, this work opens the door to the possibility of models that can continually improve in both axes.

NeurIPS Conference 2023 Conference Paper

Learning to Reason and Memorize with Self-Notes

• Jack Lanchantin
• Shubham Toshniwal
• Jason Weston
• Arthur Szlam
• Sainbayar Sukhbaatar

Large language models have been shown to struggle with multi-step reasoning, and do not retain previous reasoning steps for future use. We propose a simple method for solving both of these problems by allowing the model to take Self-Notes. Unlike recent chain-of-thought or scratchpad approaches, the model can deviate from the input context at any time to explicitly think and write down its thoughts. This allows the model to perform reasoning on the fly as it reads the context and even integrate previous reasoning steps, thus enhancing its memory with useful information and enabling multi-step reasoning. Experiments across a wide variety of tasks demonstrate that our method can outperform chain-of-thought and scratchpad methods by taking Self-Notes that interleave the input text.

NeurIPS Conference 2021 Conference Paper

Hash Layers For Large Sparse Models

• Stephen Roller
• Sainbayar Sukhbaatar
• Arthur Szlam
• Jason Weston

We investigate the training of sparse layers that use different parameters for different inputs based on hashing in large Transformer models. Specifically, we modify the feedforward layer to hash to different sets of weights depending on the current token, over all tokens in the sequence. We show that this procedure either outperforms or is competitive with learning-to-route mixture-of-expert methods such as Switch Transformers and BASE Layers, while requiring no routing parameters or extra terms in the objective function such as a load balancing loss, and no sophisticated assignment algorithm. We study the performance of different hashing techniques, hash sizes and input features, and show that balanced and random hashes focused on the most local features work best, compared to either learning clusters or using longer-range context. We show our approach works well both on large language modeling and dialogue tasks, and on downstream fine-tuning tasks.

ICML Conference 2021 Conference Paper

Not All Memories are Created Equal: Learning to Forget by Expiring

• Sainbayar Sukhbaatar
• Da Ju
• Spencer Poff
• Stephen Roller
• Arthur Szlam
• Jason Weston
• Angela Fan

Attention mechanisms have shown promising results in sequence modeling tasks that require long-term memory. Recent work investigated mechanisms to reduce the computational cost of preserving and storing memories. However, not all content in the past is equally important to remember. We propose Expire-Span, a method that learns to retain the most important information and expire the irrelevant information. This forgetting of memories enables Transformers to scale to attend over tens of thousands of previous timesteps efficiently, as not all states from previous timesteps are preserved. We demonstrate that Expire-Span can help models identify and retain critical information and show it can achieve strong performance on reinforcement learning tasks specifically designed to challenge this functionality. Next, we show that Expire-Span can scale to memories that are tens of thousands in size, setting a new state of the art on incredibly long context tasks such as character-level language modeling and a frame-by-frame moving objects task. Finally, we analyze the efficiency of Expire-Span compared to existing approaches and demonstrate that it trains faster and uses less memory.

AAAI Conference 2020 Conference Paper

Generating Interactive Worlds with Text

• Angela Fan
• Jack Urbanek
• Pratik Ringshia
• Emily Dinan
• Emma Qian
• Siddharth Karamcheti
• Shrimai Prabhumoye
• Douwe Kiela

Procedurally generating cohesive and interesting game environments is challenging and time-consuming. In order for the relationships between the game elements to be natural, common-sense has to be encoded into arrangement of the elements. In this work, we investigate a machine learning approach for world creation using content from the multiplayer text adventure game environment LIGHT (Urbanek et al. 2019). We introduce neural network based models to compositionally arrange locations, characters, and objects into a coherent whole. In addition to creating worlds based on existing elements, our models can generate new game content. Humans can also leverage our models to interactively aid in worldbuilding. We show that the game environments created with our approach are cohesive, diverse, and preferred by human evaluators compared to other machine learning based world construction algorithms.

ICLR Conference 2020 Conference Paper

Neural Text Generation With Unlikelihood Training

• Sean Welleck
• Ilia Kulikov
• Stephen Roller
• Emily Dinan
• Kyunghyun Cho
• Jason Weston

Neural text generation is a key tool in natural language applications, but it is well known there are major problems at its core. In particular, standard likelihood training and decoding leads to dull and repetitive outputs. While some post-hoc fixes have been proposed, in particular top-k and nucleus sampling, they do not address the fact that the token-level probabilities predicted by the model are poor. In this paper we show that the likelihood objective itself is at fault, resulting in a model that assigns too much probability to sequences containing repeats and frequent words, unlike those from the human training distribution. We propose a new objective, unlikelihood training, which forces unlikely generations to be assigned lower probability by the model. We show that both token and sequence level unlikelihood training give less repetitive, less dull text while maintaining perplexity, giving superior generations using standard greedy or beam search. According to human evaluations, our approach with standard beam search also outperforms the currently popular decoding methods of nucleus sampling or beam blocking, thus providing a strong alternative to existing techniques.

ICLR Conference 2020 Conference Paper

Poly-encoders: Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring

• Samuel Humeau 0001
• Kurt Shuster 0001
• Marie-Anne Lachaux
• Jason Weston

The use of deep pre-trained transformers has led to remarkable progress in a number of applications (Devlin et al., 2018). For tasks that make pairwise comparisons between sequences, matching a given input with a corresponding label, two approaches are common: Cross-encoders performing full self-attention over the pair and Bi-encoders encoding the pair separately. The former often performs better, but is too slow for practical use. In this work, we develop a new transformer architecture, the Poly-encoder, that learns global rather than token level self-attention features. We perform a detailed comparison of all three approaches, including what pre-training and fine-tuning strategies work best. We show our models achieve state-of-the-art results on four tasks; that Poly-encoders are faster than Cross-encoders and more accurate than Bi-encoders; and that the best results are obtained by pre-training on large datasets similar to the downstream tasks.

AAAI Conference 2018 Conference Paper

StarSpace: Embed All The Things!

• Ledell Wu
• Adam Fisch
• Sumit Chopra
• Keith Adams
• Antoine Bordes
• Jason Weston

We present StarSpace, a general-purpose neural embedding model that can solve a wide variety of problems: labeling tasks such as text classification, ranking tasks such as information retrieval/web search, collaborative filtering-based or content-based recommendation, embedding of multirelational graphs, and learning word, sentence or document level embeddings. In each case the model works by embedding those entities comprised of discrete features and comparing them against each other – learning similarities dependent on the task. Empirical results on a number of tasks show that StarSpace is highly competitive with existing methods, whilst also being generally applicable to new cases where those methods are not.

ICLR Conference 2017 Conference Paper

Learning End-to-End Goal-Oriented Dialog

• Antoine Bordes
• Y-Lan Boureau
• Jason Weston

Traditional dialog systems used in goal-oriented applications require a lot of domain-specific handcrafting, which hinders scaling up to new domains. End- to-end dialog systems, in which all components are trained from the dialogs themselves, escape this limitation. But the encouraging success recently obtained in chit-chat dialog may not carry over to goal-oriented settings. This paper proposes a testbed to break down the strengths and shortcomings of end-to-end dialog systems in goal-oriented applications. Set in the context of restaurant reservation, our tasks require manipulating sentences and symbols, so as to properly conduct conversations, issue API calls and use the outputs of such calls. We show that an end-to-end dialog system based on Memory Networks can reach promising, yet imperfect, performance and learn to perform non-trivial operations. We confirm those results by comparing our system to a hand-crafted slot-filling baseline on data from the second Dialog State Tracking Challenge (Henderson et al., 2014a). We show similar result patterns on data extracted from an online concierge service.

NeurIPS Conference 2016 Conference Paper

Dialog-based Language Learning

• Jason Weston

A long-term goal of machine learning research is to build an intelligent dialog agent. Most research in natural language understanding has focused on learning from fixed training sets of labeled data, with supervision either at the word level (tagging, parsing tasks) or sentence level (question answering, machine translation). This kind of supervision is not realistic of how humans learn, where language is both learned by, and used for, communication. In this work, we study dialog-based language learning, where supervision is given naturally and implicitly in the response of the dialog partner during the conversation. We study this setup in two domains: the bAbI dataset of (Weston et al. , 2015) and large-scale question answering from (Dodge et al. , 2015). We evaluate a set of baseline learning strategies on these tasks, and show that a novel model incorporating predictive lookahead is a promising approach for learning from a teacher's response. In particular, a surprising result is that it can learn to answer questions correctly without any reward-based supervision at all.

ICLR Conference 2016 Conference Paper

The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations

• Felix Hill
• Antoine Bordes
• Sumit Chopra
• Jason Weston

We introduce a new test of how well language models capture meaning in children's books. Unlike standard language modelling benchmarks, it distinguishes the task of predicting syntactic function words from that of predicting lower-frequency words, which carry greater semantic content. We compare a range of state-of-the-art models, each with a different way of encoding what has been previously read. We show that models which store explicit representations of long-term contexts outperform state-of-the-art neural language models at predicting semantic content words, although this advantage is not observed for syntactic function words. Interestingly, we find that the amount of text encoded in a single memory representation is highly influential to the performance: there is a sweet-spot, not too big and not too small, between single words and full sentences that allows the most meaningful information in a text to be effectively retained and recalled. Further, the attention over such window-based memories can be trained effectively through self-supervision. We then assess the generality of this principle by applying it to the CNN QA benchmark, which involves identifying named entities in paraphrased summaries of news articles, and achieve state-of-the-art performance.

NeurIPS Conference 2015 Conference Paper

End-To-End Memory Networks

• Sainbayar Sukhbaatar
• Arthur Szlam
• Jason Weston
• Rob Fergus

We introduce a neural network with a recurrent attention model over a possibly large external memory. The architecture is a form of Memory Network (Weston et al. , 2015) but unlike the model in that work, it is trained end-to-end, and hence requires significantly less supervision during training, making it more generally applicable in realistic settings. It can also be seen as an extension of RNNsearch to the case where multiple computational steps (hops) are performed per output symbol. The flexibility of the model allows us to apply it to tasks as diverse as (synthetic) question answering and to language modeling. For the former our approach is competitive with Memory Networks, but with less supervision. For the latter, on the Penn TreeBank and Text8 datasets our approach demonstrates comparable performance to RNNs and LSTMs. In both cases we show that the key concept of multiple computational hops yields improved results.

ICLR Conference 2015 Conference Paper

Memory Networks

• Jason Weston
• Sumit Chopra
• Antoine Bordes

We describe a new class of learning models called memory networks. Memory networks reason with inference components combined with a long-term memory component; they learn how to use these jointly. The long-term memory can be read and written to, with the goal of using it for prediction. We investigate these models in the context of question answering (QA) where the long-term memory effectively acts as a (dynamic) knowledge base, and the output is a textual response. We evaluate them on a large-scale QA task, and a smaller, but more complex, toy task generated from a simulated world. In the latter, we show the reasoning power of such models by chaining multiple supporting sentences to answer questions that require understanding the intension of verbs.

ICML Conference 2014 Conference Paper

Affinity Weighted Embedding

• Jason Weston
• Ron J. Weiss
• Hector Yee

Supervised linear embedding models like Wsabie (Weston et al. , 2011) and supervised semantic indexing (Bai et al. , 2010) have proven successful at ranking, recommendation and annotation tasks. However, despite being scalable to large datasets they do not take full advantage of the extra data due to their linear nature, and we believe they typically underfit. We propose a new class of models which aim to provide improved performance while retaining many of the benefits of the existing class of embedding models. Our approach works by reweighting each component of the embedding of features and labels with a potentially nonlinear affinity function. We describe several variants of the family, and show its usefulness on several datasets.

JMLR Journal 2014 Journal Article

Training Highly Multiclass Classifiers

• Maya R. Gupta
• Samy Bengio
• Jason Weston

Classification problems with thousands or more classes often have a large range of class-confusabilities, and we show that the more-confusable classes add more noise to the empirical loss that is minimized during training. We propose an online solution that reduces the effect of highly confusable classes in training the classifier parameters, and focuses the training on pairs of classes that are easier to differentiate at any given time in the training. We also show that the adagrad method, recently proposed for automatically decreasing step sizes for convex stochastic gradient descent, can also be profitably applied to the nonconvex joint training of supervised dimensionality reduction and linear classifiers as done in Wsabie. Experiments on ImageNet benchmark data sets and proprietary image recognition problems with 15,000 to 97,000 classes show substantial gains in classification accuracy compared to one-vs- all linear SVMs and Wsabie. [abs] [ pdf ][ bib ] &copy JMLR 2014. ( edit, beta )

ICML Conference 2013 Conference Paper

Label Partitioning For Sublinear Ranking

• Jason Weston
• Ameesh Makadia
• Hector Yee

We consider the case of ranking a very large set of labels, items, or documents, which is common to information retrieval, recommendation, and large-scale annotation tasks. We present a general approach for converting an algorithm which has linear time in the size of the set to a sublinear one via label partitioning. Our method consists of learning an input partition and a label assignment to each partition of the space such that precision at k is optimized, which is the loss function of interest in this setting. Experiments on large-scale ranking and recommendation tasks show that our method not only makes the original linear time algorithm computationally tractable, but can also improve its performance.

NeurIPS Conference 2013 Conference Paper

Translating Embeddings for Modeling Multi-relational Data

• Antoine Bordes
• Nicolas Usunier
• Alberto Garcia-Duran
• Jason Weston
• Oksana Yakhnenko

We consider the problem of embedding entities and relationships of multi-relational data in low-dimensional vector spaces. Our objective is to propose a canonical model which is easy to train, contains a reduced number of parameters and can scale up to very large databases. Hence, we propose, TransE, a method which models relationships by interpreting them as translations operating on the low-dimensional embeddings of the entities. Despite its simplicity, this assumption proves to be powerful since extensive experiments show that TransE significantly outperforms state-of-the-art methods in link prediction on two knowledge bases. Besides, it can be successfully trained on a large scale data set with 1M entities, 25k relationships and more than 17M training samples.

ICML Conference 2012 Conference Paper

Latent Collaborative Retrieval

• Jason Weston
• Chong Wang
• Ron J. Weiss
• Adam Berenzweig

UAI Conference 2012 Conference Paper

Latent Structured Ranking

• Jason Weston
• John Blitzer

Many latent (factorized) models have been proposed for recommendation tasks like collaborative filtering and for ranking tasks like document or image retrieval and annotation. Common to all those methods is that during inference the items are scored independently by their similarity to the query in the latent embedding space. The structure of the ranked list (i.e. considering the set of items returned as a whole) is not taken into account. This can be a problem because the set of top predictions can be either too diverse (contain results that contradict each other) or are not diverse enough. In this paper we introduce a method for learning latent structured rankings that improves over existing methods by providing the right blend of predictions at the top of the ranked list. Particular emphasis is put on making this method scalable. Empirical results on large scale image annotation and music recommendation tasks show improvements over existing approaches.

AAAI Conference 2011 Conference Paper

Learning Structured Embeddings of Knowledge Bases

• Antoine Bordes
• Jason Weston
• Ronan Collobert
• Yoshua Bengio

Many Knowledge Bases (KBs) are now readily available and encompass colossal quantities of information thanks to either a long-term funding effort (e. g. WordNet, OpenCyc) or a collaborative process (e. g. Freebase, DBpedia). However, each of them is based on a different rigid symbolic framework which makes it hard to use their data in other systems. It is unfortunate because such rich structured knowledge might lead to a huge leap forward in many other areas of AI like natural language processing (word-sense disambiguation, natural language understanding, .. .), vision (scene classification, image semantic annotation, .. .) or collaborative filtering. In this paper, we present a learning process based on an innovative neural network architecture designed to embed any of these symbolic representations into a more flexible continuous vector space in which the original knowledge is kept and enhanced. These learnt embeddings would allow data from any KB to be easily used in recent machine learning methods for prediction and information retrieval. We illustrate our method on WordNet and Freebase and also present a way to adapt it to knowledge extraction from raw text.

JMLR Journal 2011 Journal Article

Natural Language Processing (Almost) from Scratch

• Ronan Collobert
• Jason Weston
• Léon Bottou
• Michael Karlen
• Koray Kavukcuoglu
• Pavel Kuksa

We propose a unified neural network architecture and learning algorithm that can be applied to various natural language processing tasks including part-of-speech tagging, chunking, named entity recognition, and semantic role labeling. This versatility is achieved by trying to avoid task-specific engineering and therefore disregarding a lot of prior knowledge. Instead of exploiting man-made input features carefully optimized for each task, our system learns internal representations on the basis of vast amounts of mostly unlabeled training data. This work is then used as a basis for building a freely available tagging system with good performance and minimal computational requirements. [abs] [ pdf ][ bib ] &copy JMLR 2011. ( edit, beta )

IJCAI Conference 2011 Conference Paper

WSABIE: Scaling Up to Large Vocabulary Image Annotation

• Jason Weston
• Samy Bengio
• Nicolas Usunier

Image annotation datasets are becoming larger and larger, with tens of millions of images and tens of thousands of possible annotations. We propose a strongly performing method that scales to such datasets by simultaneously learning to optimize precision at the top of the ranked list of annotations for a given image and learning a low-dimensional joint embedding space for both images and annotations. Our method, called Wsabie, both outperforms several baseline methods and is faster and consumes less memory.

NeurIPS Conference 2010 Conference Paper

Label Embedding Trees for Large Multi-Class Tasks

• Samy Bengio
• Jason Weston
• David Grangier

Multi-class classification becomes challenging at test time when the number of classes is very large and testing against every possible class can become computationally infeasible. This problem can be alleviated by imposing (or learning) a structure over the set of classes. We propose an algorithm for learning a tree-structure of classifiers which, by optimizing the overall tree loss, provides superior accuracy to existing tree labeling methods. We also propose a method that learns to embed labels in a low dimensional space that is faster than non-embedding approaches and has superior accuracy to existing embedding approaches. Finally we combine the two ideas resulting in the label embedding tree that outperforms alternative methods including One-vs-Rest while being orders of magnitude faster.

ICML Conference 2010 Conference Paper

Label Ranking under Ambiguous Supervision for Learning Semantic Correspondences

• Antoine Bordes
• Nicolas Usunier
• Jason Weston

ICML Conference 2009 Conference Paper

Curriculum learning

• Yoshua Bengio
• Jérôme Louradour
• Ronan Collobert
• Jason Weston

Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them "curriculum learning". In the context of recent research studying the difficulty of training in the presence of non-convex training criteria (for deep deterministic and stochastic neural networks), we explore curriculum learning in various set-ups. The experiments show that significant improvements in generalization can be achieved. We hypothesize that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and, in the case of non-convex criteria, on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions).

ICML Conference 2009 Conference Paper

Deep learning from temporal coherence in video

• Hossein Mobahi
• Ronan Collobert
• Jason Weston

This work proposes a learning method for deep architectures that takes advantage of sequential data, in particular from the temporal coherence that naturally exists in unlabeled video recordings. That is, two successive frames are likely to contain the same object or objects. This coherence is used as a supervisory signal over the unlabeled data, and is used to improve the performance on a supervised task of interest. We demonstrate the effectiveness of this method on some pose invariant object and face recognition tasks.

NeurIPS Conference 2009 Conference Paper

Polynomial Semantic Indexing

• Bing Bai
• Jason Weston
• David Grangier
• Ronan Collobert
• Kunihiko Sadamasa
• Yanjun Qi
• Corinna Cortes
• Mehryar Mohri

We present a class of nonlinear (polynomial) models that are discriminatively trained to directly map from the word content in a query-document or document-document pair to a ranking score. Dealing with polynomial models on word features is computationally challenging. We propose a low rank (but diagonal preserving) representation of our polynomial models to induce feasible memory and computation requirements. We provide an empirical study on retrieval tasks based on Wikipedia documents, where we obtain state-of-the-art performance while providing realistically scalable methods.

ICML Conference 2008 Conference Paper

A unified architecture for natural language processing: deep neural networks with multitask learning

• Ronan Collobert
• Jason Weston

We describe a single convolutional neural network architecture that, given a sentence, outputs a host of language processing predictions: part-of-speech tags, chunks, named entity tags, semantic roles, semantically similar words and the likelihood that the sentence makes sense (grammatically and semantically) using a language model. The entire network is trained jointly on all these tasks using weight-sharing, an instance of multitask learning . All the tasks use labeled data except the language model which is learnt from unlabeled text and represents a novel form of semi-supervised learning for the shared tasks. We show how both multitask learning and semi-supervised learning improve the generalization of the shared tasks, resulting in state-of-the-art-performance.

ICML Conference 2008 Conference Paper

Deep learning via semi-supervised embedding

• Jason Weston
• Frédéric Ratle
• Ronan Collobert

ICML Conference 2008 Conference Paper

Large scale manifold transduction

• Michael Karlen
• Jason Weston
• Ayse Erkan
• Ronan Collobert

JMLR Journal 2007 Journal Article

Multi-class Protein Classification Using Adaptive Codes

• Iain Melvin
• Eugene Ie
• Jason Weston
• William Stafford Noble
• Christina Leslie

Predicting a protein's structural class from its amino acid sequence is a fundamental problem in computational biology. Recent machine learning work in this domain has focused on developing new input space representations for protein sequences, that is, string kernels, some of which give state-of-the-art performance for the binary prediction task of discriminating between one class and all the others. However, the underlying protein classification problem is in fact a huge multi-class problem, with over 1000 protein folds and even more structural subcategories organized into a hierarchy. To handle this challenging many-class problem while taking advantage of progress on the binary problem, we introduce an adaptive code approach in the output space of one-vs-the-rest prediction scores. Specifically, we use a ranking perceptron algorithm to learn a weighting of binary classifiers that improves multi-class prediction with respect to a fixed set of output codes. We use a cross-validation set-up to generate output vectors for training, and we define codes that capture information about the protein structural hierarchy. Our code weighting approach significantly improves on the standard one-vs-all method for two difficult multi-class protein classification problems: remote homology detection and fold recognition. Our algorithm also outperforms a previous code learning approach due to Crammer and Singer, trained here using a perceptron, when the dimension of the code vectors is high and the number of classes is large. Finally, we compare against PSI-BLAST, one of the most widely used methods in protein sequence analysis, and find that our method strongly outperforms it on every structure classification problem that we consider. Supplementary data and source code are available at http://www.cs.columbia.edu/compbio/adaptive. [abs] [ pdf ][ bib ] &copy JMLR 2007. ( edit, beta )

ICML Conference 2007 Conference Paper

Solving multiclass support vector machines with LaRank

• Antoine Bordes
• Léon Bottou
• Patrick Gallinari
• Jason Weston

JMLR Journal 2007 Journal Article

The Need for Open Source Software in Machine Learning

• Sören Sonnenburg
• Mikio L. Braun
• Cheng Soon Ong
• Samy Bengio
• Leon Bottou
• Geoffrey Holmes
• Yann LeCun
• Klaus-Robert Müller

Open source tools have recently reached a level of maturity which makes them suitable for building large-scale real-world systems. At the same time, the field of machine learning has developed a large body of powerful learning algorithms for diverse applications. However, the true potential of these methods is not used, since existing implementations are not openly shared, resulting in software with low usability, and weak interoperability. We argue that this situation can be significantly improved by increasing incentives for researchers to publish their software under an open source model. Additionally, we outline the problems authors are faced with when trying to publish algorithmic implementations of machine learning methods. We believe that a resource of peer reviewed software accompanied by short articles would be highly valuable to both the machine learning and the general scientific community. [abs] [ pdf ][ bib ] &copy JMLR 2007. ( edit, beta )

ICML Conference 2006 Conference Paper

Inference with the Universum

• Jason Weston
• Ronan Collobert
• Fabian H. Sinz
• Léon Bottou
• Vladimir Vapnik

JMLR Journal 2006 Journal Article

Large Scale Transductive SVMs

• Ronan Collobert
• Fabian Sinz
• Jason Weston
• Léon Bottou

We show how the concave-convex procedure can be applied to transductive SVMs, which traditionally require solving a combinatorial search problem. This provides for the first time a highly scalable algorithm in the nonlinear case. Detailed experiments verify the utility of our approach. Software is available at http://www.kyb.tuebingen.mpg.de/bs/people/fabee/transduction.html. [abs] [ pdf ][ bib ] &copy JMLR 2006. ( edit, beta )

ICML Conference 2006 Conference Paper

Trading convexity for scalability

• Ronan Collobert
• Fabian H. Sinz
• Jason Weston
• Léon Bottou

Convex learning algorithms, such as Support Vector Machines (SVMs), are often seen as highly desirable because they offer strong practical properties and are amenable to theoretical analysis. However, in this work we show how non-convexity can provide scalability advantages over convexity. We show how concave-convex programming can be applied to produce ( i ) faster SVMs where training errors are no longer support vectors, and ( ii ) much faster Transductive SVMs.

ICML Conference 2005 Conference Paper

A general regression technique for learning transductions

• Corinna Cortes
• Mehryar Mohri
• Jason Weston

JMLR Journal 2005 Journal Article

Fast Kernel Classifiers with Online and Active Learning

• Antoine Bordes
• Seyda Ertekin
• Jason Weston
• Léon Bottou

Very high dimensional learning systems become theoretically possible when training examples are abundant. The computing cost then becomes the limiting factor. Any efficient learning algorithm should at least take a brief look at each example. But should all examples be given equal attention? This contribution proposes an empirical answer. We first present an online SVM algorithm based on this premise. LASVM yields competitive misclassification rates after a single pass over the training examples, outspeeding state-of-the-art SVM solvers. Then we show how active example selection can yield faster training, higher accuracies, and simpler models, using only a fraction of the training example labels. [abs] [ pdf ][ bib ] &copy JMLR 2005. ( edit, beta )

ICML Conference 2005 Conference Paper

Multi-class protein fold recognition using adaptive codes

• Eugene Ie
• Jason Weston
• William Stafford Noble
• Christina S. Leslie

NeurIPS Conference 2004 Conference Paper

Breaking SVM Complexity with Cross-Training

• Léon Bottou
• Jason Weston
• Gökhan Bakir

We propose to selectively remove examples from the training set using probabilistic estimates related to editing algorithms (Devijver and Kittler, 1982). This heuristic procedure aims at creating a separable distribution of training examples with minimal impact on the position of the decision boundary. It breaks the linear dependency between the number of SVs and the number of training examples, and sharply reduces the complexity of SVMs during both the training and prediction stages.

NeurIPS Conference 2003 Conference Paper

Learning to Find Pre-Images

• Jason Weston
• Bernhard Schölkopf
• Gökhan Bakir

We consider the problem of reconstructing patterns from a feature map. Learning algorithms using kernels to operate in a reproducing kernel Hilbert space (RKHS) express their solutions in terms of input points mapped into the RKHS. We introduce a technique based on kernel princi- pal component analysis and regression to reconstruct corresponding pat- terns in the input space (aka pre-images) and review its performance in several applications requiring the construction of pre-images. The intro- duced technique avoids difficult and/or unstable numerical optimization, is easy to implement and, unlike previous methods, permits the compu- tation of pre-images in discrete input spaces.

NeurIPS Conference 2003 Conference Paper

Learning with Local and Global Consistency

• Dengyong Zhou
• Olivier Bousquet
• Thomas Lal
• Jason Weston
• Bernhard Schölkopf

We consider the general problem of learning from labeled and unlabeled data, which is often called semi-supervised learning or transductive in- ference. A principled approach to semi-supervised learning is to design a classifying function which is suf(cid: 2)ciently smooth with respect to the intrinsic structure collectively revealed by known labeled and unlabeled points. We present a simple algorithm to obtain such a smooth solution. Our method yields encouraging experimental results on a number of clas- si(cid: 2)cation problems and demonstrates effective use of unlabeled data.

NeurIPS Conference 2003 Conference Paper

Prediction on Spike Data Using Kernel Algorithms

• Jan Eichhorn
• Andreas Tolias
• Alexander Zien
• Malte Kuss
• Jason Weston
• Nikos Logothetis
• Bernhard Schölkopf
• Carl Rasmussen

We report and compare the performance of different learning algorithms based on data from cortical recordings. The task is to predict the orienta- tion of visual stimuli from the activity of a population of simultaneously recorded neurons. We compare several ways of improving the coding of the input (i. e. , the spike data) as well as of the output (i. e. , the orienta- tion), and report the results obtained using different kernel algorithms.

NeurIPS Conference 2003 Conference Paper

Ranking on Data Manifolds

• Dengyong Zhou
• Jason Weston
• Arthur Gretton
• Olivier Bousquet
• Bernhard Schölkopf

The Google search engine has enjoyed huge success with its web page ranking algorithm, which exploits global, rather than local, hyperlink structure of the web using random walks. Here we propose a simple universal ranking algorithm for data lying in the Euclidean space, such as text or image data. The core idea of our method is to rank the data with respect to the intrinsic manifold structure collectively revealed by a great amount of data. Encouraging experimental results from synthetic, image, and text data illustrate the validity of our method.

NeurIPS Conference 2003 Conference Paper

Semi-supervised Protein Classification Using Cluster Kernels

• Jason Weston
• Dengyong Zhou
• André Elisseeff
• William Noble
• Christina Leslie

A key issue in supervised protein classification is the representation of in- put sequences of amino acids. Recent work using string kernels for pro- tein data has achieved state-of-the-art classification performance. How- ever, such representations are based only on labeled data — examples with known 3D structures, organized into structural classes — while in practice, unlabeled data is far more plentiful. In this work, we de- velop simple and scalable cluster kernel techniques for incorporating un- labeled data into the representation of protein sequences. We show that our methods greatly improve the classification performance of string ker- nels and outperform standard approaches for using unlabeled data, such as adding close homologs of the positive examples to the training data. We achieve equal or superior performance to previously presented cluster kernel methods while achieving far greater computational efficiency.

JMLR Journal 2003 Journal Article

Use of the Zero-Norm with Linear Models and Kernel Methods (Kernel Machines Section)

• Jason Weston
• André Elisseeff
• Bernhard Schölkopf
• Mike Tipping

We explore the use of the so-called zero-norm of the parameters of linear models in learning. Minimization of such a quantity has many uses in a machine learning context: for variable or feature selection, minimizing training error and ensuring sparsity in solutions. We derive a simple but practical method for achieving these goals and discuss its relationship to existing techniques of minimizing the zero-norm. The method boils down to implementing a simple modification of vanilla SVM, namely via an iterative multiplicative rescaling of the training data. Applications we investigate which aid our discussion include variable and feature selection on biological microarray data, and multicategory classification.

NeurIPS Conference 2002 Conference Paper

Cluster Kernels for Semi-Supervised Learning

• Olivier Chapelle
• Jason Weston
• Bernhard Schölkopf

We propose a framework to incorporate unlabeled data in kernel classifier, based on the idea that two points in the same cluster are more likely to have the same label. This is achieved by modifying the eigenspectrum of the kernel matrix. Experimental results assess the validity of this approach.

NeurIPS Conference 2002 Conference Paper

Kernel Dependency Estimation

• Jason Weston
• Olivier Chapelle
• Vladimir Vapnik
• André Elisseeff
• Bernhard Schölkopf

We consider the learning problem of finding a dependency between a general class of objects and another, possibly different, general class of objects. The objects can be for example: vectors, images, strings, trees or graphs. Such a task is made possible by employing similarity measures in both input and output spaces using ker(cid: 173) nel functions, thus embedding the objects into vector spaces. We experimentally validate our approach on several tasks: mapping strings to strings, pattern recognition, and reconstruction from par(cid: 173) tial images.

NeurIPS Conference 2002 Conference Paper

Mismatch String Kernels for SVM Protein Classification

• Eleazar Eskin
• Jason Weston
• William Noble
• Christina Leslie

We introduce a class of string kernels, called mismatch kernels, for use with support vector machines (SVMs) in a discriminative approach to the protein classification problem. These kernels measure sequence sim- ilarity based on shared occurrences of
-length subsequences, counted with up to mismatches, and do not rely on any generative model for the positive training sequences. We compute the kernels efficiently using a mismatch tree data structure and report experiments on a benchmark SCOP dataset, where we show that the mismatch kernel used with an SVM classifier performs as well as the Fisher kernel, the most success- ful method for remote homology detection, while achieving considerable computational savings.

NeurIPS Conference 2001 Conference Paper

A kernel method for multi-labelled classification

• André Elisseeff
• Jason Weston

This article presents a Support Vector Machine (SVM) like learning sys- tem to handle multi-label problems. Such problems are usually decom- posed into many two-class problems but the expressive power of such a system can be weak [5, 7]. We explore a new direct approach. It is based on a large margin ranking system that shares a lot of common proper- ties with SVMs. We tested it on a Yeast gene functional classification problem with positive results.

NeurIPS Conference 2000 Conference Paper

Feature Selection for SVMs

• Jason Weston
• Sayan Mukherjee
• Olivier Chapelle
• Massimiliano Pontil
• Tomaso Poggio
• Vladimir Vapnik

We introduce a method of feature selection for Support Vector Machines. The method is based upon finding those features which minimize bounds on the leave-one-out error. This search can be efficiently performed via gradient descent. The resulting algorithms are shown to be superior to some standard feature selection algorithms on both toy data and real-life problems of face recognition, pedestrian detection and analyzing DNA micro array data.

NeurIPS Conference 2000 Conference Paper

Vicinal Risk Minimization

• Olivier Chapelle
• Jason Weston
• Léon Bottou
• Vladimir Vapnik

The Vicinal Risk Minimization principle establishes a bridge between generative models and methods derived from the Structural Risk Mini(cid: 173) mization Principle such as Support Vector Machines or Statistical Reg(cid: 173) ularization. We explain how VRM provides a framework which inte(cid: 173) grates a number of existing algorithms, such as Parzen windows, Support Vector Machines, Ridge Regression, Constrained Logistic Classifiers and Tangent-Prop. We then show how the approach implies new algorithm(cid: 173) s for solving problems usually associated with generative models. New algorithms are described for dealing with pattern recognition problems with very different pattern distributions and dealing with unlabeled data. Preliminary empirical results are presented.

NeurIPS Conference 1999 Conference Paper

Invariant Feature Extraction and Classification in Kernel Spaces

• Sebastian Mika
• Gunnar Rätsch
• Jason Weston
• Bernhard Schölkopf
• Alex Smola
• Klaus-Robert Müller

In hyperspectral imagery one pixel typically consists of a mixture of the reflectance spectra of several materials, where the mixture coefficients correspond to the abundances of the constituting ma(cid: 173) terials. We assume linear combinations of reflectance spectra with some additive normal sensor noise and derive a probabilistic MAP framework for analyzing hyperspectral data. As the material re(cid: 173) flectance characteristics are not know a priori, we face the problem of unsupervised linear unmixing. The incorporation of different prior information (e. g. positivity and normalization of the abun(cid: 173) dances) naturally leads to a family of interesting algorithms, for example in the noise-free case yielding an algorithm that can be understood as constrained independent component analysis (ICA). Simulations underline the usefulness of our theory.

IJCAI Conference 1999 Conference Paper

Leave-One-Out Support Vector Machines

• Jason Weston

We present a new learning algorithm for pattern recognition inspired by a recent upper bound on leave-one-out error [Jaakkola and Haussler, 1999] proved for Support Vector Machines {SVMs) [Vapnik, 1995; 1998]. The new approach directly minimizes the expression given by the bound in an attempt to minimize leave-one-out error. This gives a convex optimization problem which constructs a sparse linear classifier in feature space using the kernel technique. As such the algorithm possesses many of the same properties as SVMs. The main novelty of the algorithm is that apart from the choice of kernel, it is parameterless - the selection of the number of training errors is inherent in the algorithm and not chosen by an extra free parameter as in SVMs. First experiments using the method on benchmark datasets from the UCI repository show results similar to SVMs which have been tuned to have the best choice of parameter.

NeurIPS Conference 1999 Conference Paper

Transductive Inference for Estimating Values of Functions

• Olivier Chapelle
• Vladimir Vapnik
• Jason Weston

We introduce an algorithm for estimating the values of a function at a set of test points Xe+! ,. .. , xl+m given a set of training points (XI, YI), .. ., (xe, Ye) without estimating (as an intermediate step) the regression function. We demonstrate that this direct (transduc(cid: 173) ti ve) way for estimating values of the regression (or classification in pattern recognition) can be more accurate than the tradition(cid: 173) alone based on two steps, first estimating the function and then calculating the values of this function at the points of interest.