Author name cluster

Muhammad Awais

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

8 papers

1 author row

NeurIPS Conference 2025 Conference Paper

CG-SSL: Concept-Guided Self-Supervised Learning

Sara Atito
Josef Kittler
Imran Razzak
Muhammad Awais

Humans understand visual scenes by first capturing a global impression and then refining this understanding into distinct, object-like components. Inspired by this process, we introduce \textbf{C}oncept-\textbf{G}uided \textbf{S}elf-\textbf{S}upervised \textbf{L}earning (CG-SSL), a novel framework that brings structure and interpretability to representation learning through a curriculum of three training phases: (1) global scene encoding, (2) discovery of visual concepts via tokenised cross-attention, and (3) alignment of these concepts across views. Unlike traditional SSL methods, which simply enforce similarity between multiple augmented views of the same image, CG-SSL accounts for the fact that these views may highlight different parts of an object or scene. To address this, our method establishes explicit correspondences between views and aligns the representations of meaningful image regions. At its core, CG-SSL augments standard SSL with a lightweight decoder that learns and refines concept tokens via cross-attention with patch features. The concept tokens are trained using masked concept distillation and a feature-space reconstruction objective. A final alignment stage enforces view consistency by geometrically matching concept regions under heavy augmentation, enabling more compact, robust, and disentangled representations of scene regions. Across multiple backbone sizes, CG-SSL achieves state-of-the-art results on image segmentation benchmarks using $k$-NN and linear probes, substantially outperforming prior methods and approaching, or even surpassing, the performance of leading SSL models trained on over $100\times$ more data. Code and pretrained models will be released.

PDF Details

NeurIPS Conference 2025 Conference Paper

Dual-Stage Value-Guided Inference with Margin-Based Reward Adjustment for Fast and Faithful VLM Captioning

Ankan Deria
Adinath Dukre
Feilong Tang
Sara Atito
Sudipta Roy
Muhammad Awais
Muhammad Haris Khan
Imran Razzak

Despite significant advances in inference-time search for vision–language models (VLMs), existing approaches remain both computationally expensive and prone to unpenalized, low-confidence generations which often lead to persistent hallucinations. We introduce \textbf{Value-guided Inference with Margin-based Reward (ViMaR)}, a two-stage inference framework that improves both efficiency and output fidelity by combining a temporal-difference value model with a margin-aware reward adjustment. In the first stage, we perform a single pass to identify the highest-value caption among diverse candidates. In the second stage, we selectively refine only those segments that were overlooked or exhibit weak visual grounding, thereby eliminating frequently rewarded evaluations. A calibrated margin-based penalty discourages low-confidence continuations while preserving descriptive richness. Extensive experiments across multiple VLM architectures demonstrate that ViMaR generates captions that are significantly more reliable, factually accurate, detailed, and explanatory, while achieving over 4$\times$ speedup compared to existing value-guided methods. Specifically, we show that ViMaR trained solely on LLaVA Mistral-7B \textit{generalizes effectively to guide decoding in stronger unseen models}. To further validate this, we adapt ViMaR to steer generation in both LLaVA-OneVision-Qwen2-7B and Qwen2. 5-VL-3B, leading to consistent improvements in caption quality and demonstrating robust cross-model guidance. This cross-model generalization highlights ViMaR's flexibility and modularity, positioning it as a scalable and transferable inference-time decoding strategy. Furthermore, when ViMaR-generated captions are used for self-training, the underlying models achieve substantial gains across a broad suite of visual comprehension benchmarks, underscoring the potential of fast, accurate, and self-improving VLM pipelines. Code: https: //github. com/ankan8145/ViMaR

PDF Details

NeurIPS Conference 2025 Conference Paper

RespoDiff: Dual-Module Bottleneck Transformation for Responsible & Faithful T2I Generation

Silpa Vadakkeeveetil Sreelatha
Sauradip Nag
Muhammad Awais
Serge Belongie
Anjan Dutta

The rapid advancement of diffusion models has enabled high-fidelity and semantically rich text-to-image generation; however, ensuring fairness and safety remains an open challenge. Existing methods typically improve fairness and safety at the expense of semantic fidelity and image quality. In this work, we propose RespoDiff, a novel framework for responsible text-to-image generation that incorporates a dual-module transformation on the intermediate bottleneck representations of diffusion models. Our approach introduces two distinct learnable modules: one focused on capturing and enforcing responsible concepts, such as fairness and safety, and the other dedicated to maintaining semantic alignment with neutral prompts. To facilitate the dual learning process, we introduce a novel score-matching objective that enables effective coordination between the modules. Our method outperforms state-of-the-art methods in responsible generation by ensuring semantic alignment while optimizing both objectives without compromising image fidelity. Our approach improves responsible and semantically coherent generation by \textasciitilde20\% across diverse, unseen prompts. Moreover, it integrates seamlessly into large-scale models like SDXL, enhancing fairness and safety. The project page is available at https: //vssilpa. github. io/respodiff project page.

PDF Details

AAAI Conference 2024 Conference Paper

DTF-AT: Decoupled Time-Frequency Audio Transformer for Event Classification

Tony Alex
Sara Ahmed
Armin Mustafa
Muhammad Awais
Philip JB Jackson

Convolutional neural networks (CNNs) and Transformer-based networks have recently enjoyed significant attention for various audio classification and tagging tasks following their wide adoption in the computer vision domain. Despite the difference in information distribution between audio spectrograms and natural images, there has been limited exploration of effective information retrieval from spectrograms using domain-specific layers tailored for the audio domain. In this paper, we leverage the power of the Multi-Axis Vision Transformer (MaxViT) to create DTF-AT (Decoupled Time-Frequency Audio Transformer) that facilitates interactions across time, frequency, spatial, and channel dimensions. The proposed DTF-AT architecture is rigorously evaluated across diverse audio and speech classification tasks, consistently establishing new benchmarks for state-of-the-art (SOTA) performance. Notably, on the challenging AudioSet 2M classification task, our approach demonstrates a substantial improvement of 4.4% when the model is trained from scratch and 3.2% when the model is initialised from ImageNet-1K pretrained weights. In addition, we present comprehensive ablation studies to investigate the impact and efficacy of our proposed approach. The codebase and pretrained weights are available on https://github.com/ta012/DTFAT.git

PDF Details DOI

AAAI Conference 2024 Conference Paper

SCD-Net: Spatiotemporal Clues Disentanglement Network for Self-Supervised Skeleton-Based Action Recognition

Cong Wu
Xiao-Jun Wu
Josef Kittler
Tianyang Xu
Sara Ahmed
Muhammad Awais
Zhenhua Feng

Contrastive learning has achieved great success in skeleton-based action recognition. However, most existing approaches encode the skeleton sequences as entangled spatiotemporal representations and confine the contrasts to the same level of representation. Instead, this paper introduces a novel contrastive learning framework, namely Spatiotemporal Clues Disentanglement Network (SCD-Net). Specifically, we integrate the decoupling module with a feature extractor to derive explicit clues from spatial and temporal domains respectively. As for the training of SCD-Net, with a constructed global anchor, we encourage the interaction between the anchor and extracted clues. Further, we propose a new masking strategy with structural constraints to strengthen the contextual associations, leveraging the latest development from masked image modelling into the proposed SCD-Net. We conduct extensive evaluations on the NTU-RGB+D (60&120) and PKU-MMD (I&II) datasets, covering various downstream tasks such as action recognition, action retrieval, transfer learning, and semi-supervised learning. The experimental results demonstrate the effectiveness of our method, which outperforms the existing state-of-the-art (SOTA) approaches significantly. Our code and supplementary material can be found at https://github.com/cong-wu/SCD-Net.

PDF Details DOI

AAAI Conference 2022 Conference Paper

AXM-Net: Implicit Cross-Modal Feature Alignment for Person Re-identification

Ammarah Farooq
Muhammad Awais
Josef Kittler
Syed Safwan Khalid

Cross-modal person re-identification (Re-ID) is critical for modern video surveillance systems. The key challenge is to align cross-modality representations induced by the semantic information present for a person and ignore background information. This work presents a novel convolutional neural network (CNN) based architecture designed to learn semantically aligned cross-modal visual and textual representations. The underlying building block, named AXM-Block, is a unified multi-layer network that dynamically exploits the multi-scale knowledge from both modalities and re-calibrates each modality according to shared semantics. To complement the convolutional design, contextual attention is applied in the text branch to manipulate long-term dependencies. Moreover, we propose a unique design to enhance visual partbased feature coherence and locality information. Our framework is novel in its ability to implicitly learn aligned semantics between modalities during the feature learning stage. The unified feature learning effectively utilizes textual data as a super-annotation signal for visual representation learning and automatically rejects irrelevant information. The entire AXM-Net is trained end-to-end on CUHK-PEDES data. We report results on two tasks, person search and cross-modal Re-ID. The AXM-Net outperforms the current state-of-theart (SOTA) methods and achieves 64. 44% Rank@1 on the CUHK-PEDES test set. It also outperforms its competitors by >10% in cross-viewpoint text-to-image Re-ID scenarios on CrossRe-ID and CUHK-SYSU datasets.

PDF Details

JBHI Journal 2021 Journal Article

A Hybrid DCNN-SVM Model for Classifying Neonatal Sleep and Wake States Based on Facial Expressions in Video

Muhammad Awais
Xi Long
Bin Yin
Saadullah Farooq Abbasi
Saeed Akbarzadeh
Chunmei Lu
Xinhua Wang
Laishuan Wang

Sleep is a natural phenomenon controlled by the central nervous system. The sleep-wake pattern, which functions as an essential indicator of neurophysiological organization in the neonatal period, has profound meaning in the prediction of cognitive diseases and brain maturity. In recent years, unobtrusive sleep monitoring and automatic sleep staging have been intensively studied for adults, but much less for neonates. This work aims to investigate a novel video-based unobtrusive method for neonatal sleep-wake classification by analyzing the behavioral changes in the neonatal facial region. A hybrid model is proposed to monitor the sleep-wake patterns of human neonates. The model combines two algorithms: deep convolutional neural network (DCNN) and support vector machine (SVM), where DCNN works as a trainable feature extractor and SVM as a classifier. Data was collected from nineteen Chinese neonates at the Children's Hospital of Fudan University, Shanghai, China. The classification results are compared with the gold standard of video-electroencephalography scored by pediatric neurologists. Validations indicate that the proposed hybrid DCNN-SVM model achieved reliable performances in classifying neonatal sleep and wake states in RGB video frames (with the face region detected), with an accuracy of 93. 8 ± 2. 2% and an F1-score 0. 93 ± 0. 3.

Details DOI

JBHI Journal 2019 Journal Article

Physical Activity Classification for Elderly People in Free-Living Conditions

Muhammad Awais
Lorenzo Chiari
Espen Alexander F. Ihlen
Jorunn L. Helbostad
Luca Palmerini

Physical activity is strongly linked with mental and physical health in the elderly population and accurate monitoring of activities of daily living (ADLs) can help improve quality of life and well-being. This study presents and validates an inertial sensors-based physical activity classification system developed with older adults as the target population. The dataset was collected in free-living conditions without placing constraints on the way and order of performing ADLs. Four sensor locations (chest, lower back, wrist, and thigh) were explored to obtain the optimal number and combination of sensors by finding the best tradeoff between the system's performance and wearability. Several feature selection techniques were implemented on the feature set obtained from acceleration and angular velocity signals to classify four major ADLs (sitting, standing, walking, and lying). A support vector machine was used for the classification of the ADLs. The findings show the potential of different solutions (single sensor or multisensor) to correctly classify the ADLs of older people in free-living conditions. Considering a minimal set-up of a single sensor, the sensor worn at the L5 achieved the best performance. A two-sensor solution (L5 + thigh) achieved a better performance with respect to a single-sensor solution. By contrast, considering more than two sensors did not provide further improvements. Finally, we evaluated the computational cost of different solutions and it was shown that a feature selection step can reduce the computational cost of the system and increase the system performance in most cases. This can be helpful for real-time applications.

Details DOI