Author name cluster

Munawar Hayat

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

7 papers

2 author rows

AAAI Conference 2024 Conference Paper

Concealing Sensitive Samples against Gradient Leakage in Federated Learning

Jing Wu
Munawar Hayat
Mingyi Zhou
Mehrtash Harandi

Federated Learning (FL) is a distributed learning paradigm that enhances users' privacy by eliminating the need for clients to share raw, private data with the server. Despite the success, recent studies expose the vulnerability of FL to model inversion attacks, where adversaries reconstruct users’ private data via eavesdropping on the shared gradient information. We hypothesize that a key factor in the success of such attacks is the low entanglement among gradients per data within the batch during stochastic optimization. This creates a vulnerability that an adversary can exploit to reconstruct the sensitive data. Building upon this insight, we present a simple, yet effective defense strategy that obfuscates the gradients of the sensitive data with concealed samples. To achieve this, we propose synthesizing concealed samples to mimic the sensitive data at the gradient level while ensuring their visual dissimilarity from the actual sensitive data. Compared to the previous art, our empirical evaluations suggest that the proposed technique provides the strongest protection while simultaneously maintaining the FL performance. Code is located at https://github.com/JingWu321/DCS-2.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

FouRA: Fourier Low-Rank Adaptation

Shubhankar Borse
Shreya Kadambi
Nilesh P. Pandey
Kartikeya Bhardwaj
Viswanath Ganapathy
Sweta Priyadarshi
Risheek Garrepalli
Rafael Esteves

While Low-Rank Adaptation (LoRA) has proven beneficial for efficiently fine-tuning large models, LoRA fine-tuned text-to-image diffusion models lack diversity in the generated images, as the model tends to copy data from the observed training samples. This effect becomes more pronounced at higher values of adapter strength and for adapters with higher ranks which are fine-tuned on smaller datasets. To address these challenges, we present FouRA, a novel low-rank method that learns projections in the Fourier domain along with learning a flexible input-dependent adapter rank selection strategy. Through extensive experiments and analysis, we show that FouRA successfully solves the problems related to data copying and distribution collapse while significantly improving the generated image quality. We demonstrate that FouRA enhances the generalization of fine-tuned models thanks to its adaptive rank selection. We further show that the learned projections in the frequency domain are decorrelated and prove effective when merging multiple adapters. While FouRA is motivated for vision tasks, we also demonstrate its merits for language tasks on commonsense reasoning and GLUE benchmarks.

PDF Details DOI

EAAI Journal 2024 Journal Article

Survey: Image mixing and deleting for data augmentation

Humza Naveed
Saeed Anwar
Munawar Hayat
Kashif Javed
Ajmal Mian

Neural networks are prone to overfitting and memorizing data patterns. To avoid over-fitting and enhance their generalization and performance, various methods have been suggested in the literature, including dropout, regularization, label smoothing, etc. One such method is augmentation which introduces different types of corruption in the data to prevent the model from overfitting and to memorize patterns present in the data. A sub-area of data augmentation is image mixing and deleting. This specific type of augmentation either deletes image regions or mixes two images to hide or make particular characteristics of images confusing for the network, forcing it to emphasize the overall structure of the object in an image. Models trained with this approach have proven to perform and generalize well compared to those trained without image mixing or deleting. An added benefit that comes with this method of training is robustness against image corruption. Due to its low computational cost and recent success, researchers have proposed many image mixing and deleting techniques. We furnish an in-depth survey of image mixing and deleting techniques and provide categorization via their most distinguishing features. We initiate our discussion with some fundamental relevant concepts. Next, we present essentials, such as each category’s strengths and limitations, describing their working mechanism, basic formulations, and applications. We also discuss the general challenges and recommend possible future research directions for image mixing and deleting data augmentation techniques. Datasets and codes for evaluation are publicly available here.

Details DOI

IROS Conference 2023 Conference Paper

Real-Time Trajectory-Based Social Group Detection

Simindokht Jahangard
Munawar Hayat
Hamid Rezatofighi

Social group detection is a crucial aspect of various robotic applications, including robot navigation and human-robot interactions. To date, a range of model-based techniques have been employed to address this challenge, such as the F-formation and trajectory similarity frameworks. However, these approaches often fail to provide reliable results in crowded and dynamic scenarios. Recent advancements in this area have mainly focused on learning-based methods, such as deep neural networks that use visual content or human pose. Although visual content based methods have demonstrated promising performance on large-scale datasets, their computational complexity poses a significant barrier to their practical use in real-time applications. To address these issues, we propose a simple and efficient framework for social group detection. Our approach explores the impact of motion trajectory on social grouping and utilizes a novel, reliable, and fast data-driven method. We formulate the individuals in a scene as a graph, where the nodes are represented by LSTM-encoded trajectories and the edges are defined by the distances between each pair of tracks. Our framework employs a modified graph transformer module and graph clustering losses to detect social groups. Our experiments on the popular JRDB-Act dataset reveal noticeable improvements in performance, with relative improvements ranging from 2% to 11%. Furthermore, our framework is significantly faster, with up to 12x faster inference times compared to state-of-the-art methods under the same computation resources. These results demonstrate that our proposed method is suitable for real-time robotic applications. .

Details

EAAI Journal 2023 Journal Article

Synthesis of a six-bar mechanism for generating knee and ankle motion trajectories using deep generative neural network

Akim Kapsalyamov
Shahid Hussain
Nicholas A.T. Brown
Roland Goecke
Munawar Hayat
Prashant K. Jamwal

Robotic exoskeletons have demonstrated their effectiveness in post-stroke gait rehabilitation therapy. Nevertheless, further research is being conducted to improve existing rehabilitation exoskeletons in terms of ease-of-use and innovative design. Previously, the adaptation of linkage-based mechanisms for rehabilitation exoskeletons has been considered an option. However, finding linkage parameters that will produce the required gait trajectories using a linkage-based exoskeleton, is quite challenging. It is furthermore challenging to obtain parameters of a linkage-based mechanism designed for a gait rehabilitation task that has to produce two trajectories (for knee and ankle joints) simultaneously. In this work, we propose Deep Generative Neural Networks (DGNN) to obtain a set of optimal dimensions and parameters for the Stephenson III six-bar linkage-based gait exoskeleton. The proposed methodology demonstrates high efficacy in determining the linkage parameters for various target trajectories. The proposed framework, once trained, can accurately predict mechanism parameters to achieve two joint trajectories simultaneously. Subsequent to developing the model, walking trajectories from healthy human subjects are given to the model to determine the optimal linkage dimensions of the gait rehabilitation exoskeleton. The proposed model can be used to assist designers in quickly determining the optimized linkage dimensions of linkage-based mechanisms that can provide various target trajectories.

Details DOI

NeurIPS Conference 2021 Conference Paper

Intriguing Properties of Vision Transformers

Muhammad Muzammal Naseer
Kanchana Ranasinghe
Salman H Khan
Munawar Hayat
Fahad Shahbaz Khan
Ming-Hsuan Yang

Vision transformers (ViT) have demonstrated impressive performance across numerous machine vision tasks. These models are based on multi-head self-attention mechanisms that can flexibly attend to a sequence of image patches to encode contextual cues. An important question is how such flexibility (in attending image-wide context conditioned on a given patch) can facilitate handling nuisances in natural images e. g. , severe occlusions, domain shifts, spatial permutations, adversarial and natural perturbations. We systematically study this question via an extensive set of experiments encompassing three ViT families and provide comparisons with a high-performing convolutional neural network (CNN). We show and analyze the following intriguing properties of ViT: (a)Transformers are highly robust to severe occlusions, perturbations and domain shifts, e. g. , retain as high as 60% top-1 accuracy on ImageNet even after randomly occluding 80% of the image content. (b)The robustness towards occlusions is not due to texture bias, instead we show that ViTs are significantly less biased towards local textures, compared to CNNs. When properly trained to encode shape-based features, ViTs demonstrate shape recognition capability comparable to that of human visual system, previously unmatched in the literature. (c)Using ViTs to encode shape representation leads to an interesting consequence of accurate semantic segmentation without pixel-level supervision. (d)Off-the-shelf features from a single ViT model can be combined to create a feature ensemble, leading to high accuracy rates across a range of classification datasets in both traditional and few-shot learning paradigms. We show effective features of ViTs are due to flexible and dynamic receptive fields possible via self-attention mechanisms. Our code will be publicly released.

PDF Details

NeurIPS Conference 2019 Conference Paper

Random Path Selection for Continual Learning

Jathushan Rajasegaran
Munawar Hayat
Salman Khan
Fahad Shahbaz Khan
Ling Shao

Incremental life-long learning is a main challenge towards the long-standing goal of Artificial General Intelligence. In real-life settings, learning tasks arrive in a sequence and machine learning models must continually learn to increment already acquired knowledge. The existing incremental learning approaches fall well below the state-of-the-art cumulative models that use all training classes at once. In this paper, we propose a random path selection algorithm, called RPS-Net, that progressively chooses optimal paths for the new tasks while encouraging parameter sharing and reuse. Our approach avoids the overhead introduced by computationally expensive evolutionary and reinforcement learning based path selection strategies while achieving considerable performance gains. As an added novelty, the proposed model integrates knowledge distillation and retrospection along with the path selection strategy to overcome catastrophic forgetting. In order to maintain an equilibrium between previous and newly acquired knowledge, we propose a simple controller to dynamically balance the model plasticity. Through extensive experiments, we demonstrate that the proposed method surpasses the state-of-the-art performance on incremental learning and by utilizing parallel computation this method can run in constant time with nearly the same efficiency as a conventional deep convolutional neural network.

PDF Details