Sachindra Joshi

ICML Conference 2024 Conference Paper

BRAIn: Bayesian Reward-conditioned Amortized Inference for natural language generation from feedback

• Gaurav Pandey 0001
• Yatin Nandwani
• Tahira Naseem
• Mayank Mishra
• Guangxuan Xu
• Dinesh Raghu
• Sachindra Joshi
• Asim Munawar

Distribution matching methods for language model alignment such as Generation with Distributional Control (GDC) and Distributional Policy Gradient (DPG) have not received the same level of attention in reinforcement learning from human feedback (RLHF) as contrastive methods such as Sequence Likelihood Calibration (SLiC), Direct Preference Optimization (DPO) and its variants. We identify high variance of the gradient estimate as the primary reason for the lack of success of these methods and propose a self-normalized baseline to reduce the variance. We further generalize the target distribution in DPG, GDC and DPO by using Bayes’ rule to define the reward-conditioned posterior. The resulting approach, referred to as BRAIn - Bayesian Reward-conditioned Amortized Inference acts as a bridge between distribution matching methods and DPO and significantly outperforms prior art in summarization and Antropic HH tasks.

AAAI Conference 2021 System Paper

Bootstrapping Dialog Models from Human to Human Conversation Logs

• Pankaj Dhoolia
• Vineet Kumar
• Danish Contractor
• Sachindra Joshi

State-of-the-art commercial dialog platforms provide powerful tools to build a conversational agent. These platforms provide complete control to the dialog designer to model useragent interactions. However, a dialog designer needs to rely on domain experts to manually build the dialog model – by creating dialog flow nodes and modeling user intents. This process is laborious, time consuming and expensive and does not allow the designer to exploit human to human conversation logs effectively. In this work, we present a research prototype that can ingest human-to-human conversation logs between an end-user and an agent, and suggest user-intents and agent-responses, given a conversation context. We utilize human to human conversation logs to build two emulators: user and agent. An agent emulator models an agent response given the conversation context so far, and a user emulator outputs possible user responses. Our system is able to recommend conversational intents as well as conversation flow using emulators based on real-world data, thus making the process of designing a bot more efficient. To the best our knowledge this is the first system that enables data-driven dialog model creation by emulating users and agents.

AAAI Conference 2021 System Paper

Doc2Bot: Document grounded Bot Framework

• Kshitij Fadnis
• Pankaj Dhoolia
• Li Zhu
• Q. Vera Liao
• Steven Ross
• Nathaniel Mills
• Sachindra Joshi
• Luis Lastras

Conversational agents – or chatbots – are widely used to provide customer care and other informational support. Currently, the development of chatbots using standard frameworks requires a lot of manual crafting by subject matter experts (SMEs). On the other hand, while learning-based approaches to dialog have made significant advancements, they require training with a large volume of dialog data, which chatbot developers typically do not have access to. To tackle these challenges, we introduce DOC2BOT, a system that supports the automated construction of chatbots by digesting various forms of documents such as business manuals, HowTos, and customer support pages that organizations own. In addition to this, DOC2BOT provides a user-friendly experience to SMEs, and minimizes the effort expended by them by supporting intuitive interactions and streamlining their workflow.

AAAI Conference 2020 Conference Paper

Mask & Focus: Conversation Modelling by Learning Concepts

• Gaurav Pandey
• Dinesh Raghu
• Sachindra Joshi

Sequence to sequence models attempt to capture the correlation between all the words in the input and output sequences. While this is quite useful for machine translation where the correlation among the words is indeed quite strong, it becomes problematic for conversation modelling where the correlation is often at a much abstract level. In contrast, humans tend to focus on the essential concepts discussed in the conversation context and generate responses accordingly. In this paper, we attempt to mimic this response generating mechanism by learning the essential concepts in the context and response in an unsupervised manner. The proposed model, referred to as Mask & Focus maps the input context to a sequence of concepts which are then used to generate the response concepts. Together, the context and the response concepts generate the final response. In order to learn context concepts from the training data automatically, we mask words in the input and observe the effect of masking on response generation. We train our model to learn those response concepts that have high mutual information with respect to the context concepts, thereby guiding the model to focus on the context concepts. Mask & Focus achieves significant improvement over the existing baselines in several established metrics for dialogues.

ECAI Conference 2020 Conference Paper

Unsupervised Learning of Interpretable Dialog Models

• Dhiraj Madan
• Dinesh Raghu
• Gaurav Pandey 0001
• Sachindra Joshi

Recently several deep learning based models have been proposed for end-to-end learning of dialogs. While these models can be trained from data without the need for any additional annotations, it is hard to interpret them. On the other hand, there exist traditional state based dialog systems, where the states of the dialog are discrete and hence easy to interpret. However these states need to be handcrafted and annotated in the data. To achieve the best of both worlds, we propose Latent State Tracking Network (LSTN) using which we learn an interpretable model in unsupervised manner. The model defines a discrete latent variable at each turn of the conversation which can take a finite set of values. These variables correspond to the state of the dialog after each turn. Since the conversations are not labelled with the dialog states, we use EM algorithm to train our model in unsupervised manner. In the experiments, we show that LSTN can help achieve interpretability in dialog models with performance comparable to end-to-end approaches. This interpretability allows us to edit the model and improve the same.

AAAI Conference 2018 Conference Paper

Dialogue Act Sequence Labeling Using Hierarchical Encoder With CRF

• Harshit Kumar
• Arvind Agarwal
• Riddhiman Dasgupta
• Sachindra Joshi

Dialogue Act recognition associate dialogue acts (i. e. , semantic labels) to utterances in a conversation. The problem of associating semantic labels to utterances can be treated as a sequence labeling problem. In this work, we build a hierarchical recurrent neural network using bidirectional LSTM as a base unit and the conditional random field (CRF) as the top layer to classify each utterance into its corresponding dialogue act. The hierarchical network learns representations at multiple levels, i. e. , word level, utterance level, and conversation level. The conversation level representations are input to the CRF layer, which takes into account not only all previous utterances but also their dialogue acts, thus modeling the dependency among both, labels and utterances, an important consideration of natural dialogue. We validate our approach on two different benchmark data sets, Switchboard and Meeting Recorder Dialogue Act, and show performance improvement over the state-of-the-art methods by 2. 2% and 4. 1% absolute points, respectively. It is worth noting that the inter-annotator agreement on Switchboard data set is 84%, and our method is able to achieve the accuracy of about 79% despite being trained on the noisy data.

ICML Conference 2003 Conference Paper

Classification of Text Documents Based on Minimum System Entropy

• Raghu Krishnapuram
• Krishna Prasad Chitrapura
• Sachindra Joshi