Author name cluster

Deepak Mishra

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

6 papers

2 author rows

TMLR Journal 2026 Journal Article

Correctness-Aware Knowledge Distillation for Enhanced Student Learning

Ishan Mishra
Deepak Mishra
Jinjun Xiong

In real-world learning, students rely on their mentors for guidance but must also develop the ability to recognize and learn from their mentors' mistakes. Inspired by this mentor-critic dynamic, we propose Mentor-Critic Distillation (MCD), a novel framework for knowledge distillation in machine learning. Traditional distillation methods risk transferring both correct insights and errors from the mentor (teacher model) to the student model, which can hinder student performance. Notably, previous state-of-the-art approaches fail to account for scenarios where the teacher is incorrect, often leaving the student model vulnerable to inheriting these errors. To address this limitation, MCD introduces a weighted knowledge transfer mechanism that decouples the learning process based on the mentor's correctness. When the mentor model is correct, the student model follows the mentor's guidance with a large weight on knowledge transfer. However, when the mentor is incorrect, the student relies more on the ground truth but still learns inter-class relationships from the mentor, adjusting the weight toward task-specific losses such as cross-entropy. This mentor-critic approach ensures that the student model benefits from the mentor's expertise without inheriting its mistakes. We provide theoretical analysis proving that MCD strictly generalizes vanilla KD and guarantees reduced negative transfer. We evaluate our Mentor-Critic Distillation across diverse teacher-student configurations on benchmark datasets, including CIFAR-100, ImageNet, and MedMNIST. Notably, MCD requires no architectural modifications or additional parameters, making it a practical drop-in replacement for standard knowledge distillation. These results highlight MCD's effectiveness in optimizing knowledge transfer and its robustness across diverse domains and data regimes, particularly in data-scarce scenarios typical of specialized domains such as medical imaging.

PDF Details

JBHI Journal 2024 Journal Article

MLVICX: Multi-Level Variance-Covariance Exploration for Chest X-Ray Self-Supervised Representation Learning

Azad Singh
Vandan Gorade
Deepak Mishra

Self-supervised learning (SSL) reduces the need for manual annotation in deep learning models for medical image analysis. By learning the representations from unablelled data, self-supervised models perform well on tasks that require little to no fine-tuning. However, for medical images, like chest X-rays, characterised by complex anatomical structures and diverse clinical conditions, a need arises for representation learning techniques that encode fine-grained details while preserving the broader contextual information. In this context, we introduce MLVICX (Multi-Level Variance-Covariance Exploration for Chest X-ray Self-Supervised Representation Learning), an approach to capture rich representations in the form of embeddings from chest X-ray images. Central to our approach is a novel multi-level variance and covariance exploration strategy that effectively enables the model to detect diagnostically meaningful patterns while reducing redundancy. MLVICX promotes the retention of critical medical insights by adapting global and local contextual details and enhancing the variance and covariance of the learned embeddings. We demonstrate the performance of MLVICX in advancing self-supervised chest X-ray representation learning through comprehensive experiments. The performance enhancements we observe across various downstream tasks highlight the significance of the proposed approach in enhancing the utility of chest X-ray embeddings for precision medical diagnosis and comprehensive image analysis. For pertaining, we used the NIH-Chest X-ray dataset. Downstream tasks utilized NIH-Chest X-ray, Vinbig-CXR, RSNA pneumonia, and SIIM-ACR Pneumothorax datasets. Overall, we observe up to 3% performance gain over SOTA SSL approaches in various downstream tasks. Additionally, to demonstrate generalizability of our method, we conducted additional experiments on fundus images and observed superior performance on multiple datasets. Codes are available at GitHub.

Details DOI

JBHI Journal 2023 Journal Article

Clinically Relevant Myocardium Segmentation in Cardiac Magnetic Resonance Images

Rohit Gavirni
Divij Gupta
Deepak Mishra
Arbind Gupta
Sanjaya Viswamitra

Deep learning approaches have shown great success in myocardium region segmentation in Cardiac MR (CMR) images. However, most of these often ignore irregularities such as protrusions, breaks in contour, etc. As a result, the common practice by clinicians is to manually correct the obtained outputs for the evaluation of myocardium condition. This paper aims to make the deep learning systems capable of handling the aforementioned irregularities and satisfy desired clinical constraints, necessary for various downstream clinical analysis. We propose a refinement model which imposes structural constraints on the outputs of the existing deep learning-based myocardium segmentation methods. The complete system is a pipeline of deep neural networks where an initial network performs myocardium segmentation as accurate as possible and the refinement network removes defects from the initial output to make it suitable for clinical decision support systems. We experiment with datasets collected from four different sources and observe consistent final segmentation outputs with improvement up to 8% in Dice Coefficient and up to 18 pixels in Hausdorff Distance due to the proposed refinement model. The proposed refinement strategy leads to qualitative and quantitative improvements in the performances of all the considered segmentation networks. Our work is an important step towards the development of a fully automatic myocardium segmentation system. It can also be generalized for other tasks where the object of interest has regular structure and the defects can be modelled statistically.

Details DOI

NeurIPS Conference 2022 Conference Paper

Discovering and Overcoming Limitations of Noise-engineered Data-free Knowledge Distillation

Piyush Raikwar
Deepak Mishra

Distillation in neural networks using only the samples randomly drawn from a Gaussian distribution is possibly the most straightforward solution one can think of for the complex problem of knowledge transfer from one network (teacher) to the other (student). If successfully done, it can eliminate the requirement of teacher's training data for knowledge distillation and avoid often arising privacy concerns in sensitive applications such as healthcare. There have been some recent attempts at Gaussian noise-based data-free knowledge distillation, however, none of them offer a consistent or reliable solution. We identify the shift in the distribution of hidden layer activation as the key limiting factor, which occurs when Gaussian noise is fed to the teacher network instead of the accustomed training data. We propose a simple solution to mitigate this shift and show that for vision tasks, such as classification, it is possible to achieve a performance close to the teacher by just using the samples randomly drawn from a Gaussian distribution. We validate our approach on CIFAR10, CIFAR100, SVHN, and Food101 datasets. We further show that in situations of sparsely available original data for distillation, the proposed Gaussian noise-based knowledge distillation method can outperform the distillation using the available data with a large margin. Our work lays the foundation for further research in the direction of noise-engineered knowledge distillation using random samples.

PDF Details

AAAI Conference 2022 Short Paper

MBGRLp: Multiscale Bootstrap Graph Representation Learning on Pointcloud (Student Abstract)

Vandan Gorade
Azad Singh
Deepak Mishra

Point cloud has gained a lot of attention with the availability of large amount of point cloud data and increasing applications like city planning and self-driving cars. However, current methods, often rely on labeled information and costly processing, such as converting point cloud to voxel. We propose a self-supervised learning approach to tackle these problems, combating labelling and additional memory cost issues. Our proposed method achieves results comparable to supervised and unsupervised baselines on widely used benchmark datasets for self-supervised point cloud classification like ShapeNet, ModelNet10/40.

PDF Details

ICRA Conference 2014 Conference Paper

Sensors for micro bio robots via synthetic biology

Edward B. Steager
Denise Wong
Deepak Mishra
Ron Weiss
Vijay Kumar 0001

Microscale robots offer an unprecedented opportunity to perform tasks at resolutions approaching 1 μm, but the great majority of research to this point focuses on actuation and control. Potential applications for microrobots can be considerably expanded by integrating sensing, signal processing and feedback into the system. In this work, we demonstrate that technologies from the field of synthetic biology may be directly integrated into microrobotic systems to create cell-based programmable mobile sensors, with signal processors and memory units. Specifically, we integrate genetically engineered, ultraviolet light-sensing bacteria with magnetic microrobots, creating the first controllable biological microrobot that is capable of exploring, recording and reporting on the state of the microscale environment. We demonstrate two proof-of-concept prototypes: (a) an integrated microrobot platform that is able to sense biochemical signals, and (b) a microrobot platform that is able to deploy biosensor payloads to monitor biochemical signals, both in a biological environment. These results have important implications for integrated micro-bio-robotic systems for applications in biological engineering and research.

Details