Author name cluster

Abdulmotaleb El Saddik

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

10 papers

2 author rows

EAAI Journal 2026 Journal Article

Enhancing multimodal emotion recognition with dynamic fuzzy membership and attention fusion

Nhut Minh Nguyen
Trung Minh Nguyen
Thanh Trung Nguyen
Phuong-Nam Tran
Nhat Truong Pham
Linh Le
Alice Othmani
Abdulmotaleb El Saddik

Multimodal learning has been shown to enhance classification outcomes in speech emotion recognition (SER). Despite this advantage, multimodal approaches in SER often face key challenges, including limited robustness to uncertainty, difficulty generalizing across diverse emotional contexts, and inefficiencies in integrating heterogeneous modalities. To overcome these constraints, we propose a multimodal emotion recognition architecture, named FleSER, which leverages dynamic fuzzy membership and attention-based fusion. Unlike most previous SER studies that apply fuzzy logic at the decision level, FleSER introduces a feature-level, rule-based dynamic fuzzy membership mechanism that adaptively refines modality representations prior to fusion. The FleSER architecture leverages audio and textual modalities, employing self-modality and cross-modality attention mechanisms with the α interpolation to capture complementary emotional cues. The α interpolation-based feature fusion mechanism adaptively emphasizes the more informative modality across varying contexts, ensuring robust multimodal integration. This comprehensive design enhances recognition accuracy. We evaluate FleSER on multiple benchmark datasets, surpassing previous state-of-the-art (SOTA) approaches and demonstrating superior effectiveness in emotion recognition. Ablation studies further validate the effectiveness of each key component, including unimodal and multimodal input effectiveness, fuzzy membership functions, fusion strategies, and the projection dimension, on the performance of the FleSER architecture.

Details DOI

EAAI Journal 2026 Journal Article

Person search with deep learning

Ning Lv
Xuezhi Xiang
Yulong Qiao
Abdulmotaleb El Saddik

Person search is a crucial area of image retrieval and has diverse practical applications in real-world surveillance applications such as intelligent traffic, smart cities and so on. The objective is to locate and identify a specific pedestrian within a gallery of scene images, based on the query image. The person search task involves the localization and re-identification of target pedestrians, and can be divided into two subtasks: pedestrian detection and person re-identification. With the rapid evolution of technology, various deep learning-based person search methods have been proposed in recent years. Despite these advancements, there remains a lack of comprehensive overview over the research developments in person search. Therefore, we present an in-depth analysis within the domain. The survey begins with an exploration of fully supervised person search methods, covering the advances in framework and loss function. From the perspective of task processing, we examine the challenges such as accuracy of detection, discriminative re-identification features, relationship between detection and re-identification, background influence, and discuss how existing methods address these issues. Furthermore, we investigate weakly supervised person search, where limited annotations pose challenges. We discuss the annotation settings used in recent methods, which determine how the model will be trained. We also explore the role of clustering techniques and contextual information in addressing the annotation quality issue. The survey covers the evaluation metrics, benchmark datasets, and a performance evaluation of person search methods. Finally, we discuss the potential future directions for research in the field of person search.

Details DOI

IS Journal 2024 Journal Article

Tile-Weighted Rate-Distortion Optimized Packet Scheduling for 360° Virtual-Reality Video Streaming

Haopeng Wang
Haiwei Dong
Abdulmotaleb El Saddik

A key challenge of 360° virtual-reality (VR) video streaming is ensuring high quality with limited network bandwidth. Currently, most of the studies focus on tile-based adaptive bit-rate streaming to reduce bandwidth consumption, where resources in network nodes are not fully utilized. This article proposes a tile-weighted rate-distortion (TWRD) packet scheduling optimization system to reduce data volume and improve video quality. A multimodal spatial–temporal attention transformer is proposed to predict viewpoint with probability that is used to dynamically weight tiles and their corresponding packets. The packet scheduling problem of determining which packets should be dropped is formulated as an optimization problem solved by a dynamic programming solution. Experiment results demonstrate that the proposed method outperforms the existing methods under various conditions.

Details DOI

EAAI Journal 2024 Journal Article

Yield estimation and health assessment of temperate fruits: A modular framework

Jamil Ahmad
Wail Gueaieb
Abdulmotaleb El Saddik
Giulia De Masi
Fakhri Karray

Yield estimation is crucial for growers and agronomists to optimize crop management practices and facilitate harvest planning. However, traditional manual fruit counting and fruit health assessment activities on large fields are labor-intensive, time-consuming, and prone to errors. Computer vision-based yield estimation methods involving fruit counting and health assessment using unmanned aerial vehicles (UAVs), have gained significant attention in recent years. This study proposes an automated yield estimation and health assessment approach through UAV imaging. Our methodology comprises three main components: (1) a robust fruit detection network based on the “you only look once - neural architecture search” (YOLONAS) model, (2) a fruit health assessment module to detect diseases in individually identified fruits, and (3) a post-processing and regression module for yield quantity and quality estimation. YOLONAS is a computationally efficient and accurate object detection model trained on scale-space augmented datasets. The health assessment module includes separable multiscale convolution layers with an additive attention module. We evaluated our yield estimation approach on three publicly available datasets featuring peach, apple, and citrus trees. Results reveal that YOLONAS, trained with a scale-space augmented dataset, improves detection accuracy by 1. 2%. We also used a custom fruit disease dataset to assess the performance of the disease detection model, where we noticed that super-resolution of detected fruits with pre-trained models significantly enhances disease detection by up to 17%, especially in low-resolution fruits. Finally, we demonstrate that the proposed method can serve as a modular framework for yield quantity and quality assessment through UAVs in challenging field conditions.

Details DOI

ICRA Conference 2023 Conference Paper

CEAFFOD: Cross-Ensemble Attention-based Feature Fusion Architecture Towards a Robust and Real-time UAV-based Object Detection in Complex Scenarios

Ahmed Elhagry
Hang Dai
Abdulmotaleb El Saddik
Wail Gueaieb
Giulia De Masi

Deploying object detectors in embedded devices such as unmanned aerial vehicles (UAVs) comes with many challenges. This is due to both the UAV itself having low embedded resources in terms of computation and memory, and also due to the nature of the captured visual data with the variations in objects' scale, orientation, density, viewpoint, distribution, shape, context and others. It is crucial for the object detector to be robust with high accuracy, real-time with fast inference and light-weight to be applicable. Inspired by YOLO architecture, we propose a novel single-stage detection architecture. Our contributions are, first, feature fusion spatial pyramid pooling (FFSPP) block that applies attention-based feature fusion across both time and space utilizing the information of subsequent frames and scales in an efficient manner. Secondly, we introduce a multi-dilated attention-based cross-stage partial connection (MDACSP) block that helps in increasing the receptive field and producing per-channel modulation weights after aggregating the feature maps across their spatial domain. Third, scaled feature fusion head (SFFH) fuses both the FFSPP block features and the connected MDACSP block features specific for this head. For a more robust result across different scenarios, we perform cross-ensembling with three of the top UAV/traffic surveillance datasets: UAVDT, UA-DETRAC and VisDrone. Our ablation study shows how every contribution improves over the baseline. Our approach yielded the state-of-the-art results in all the aforementioned datasets achieving 89. 3% mAP, 93. 5% mAP, and 42. 9% mAP respectively. Testing the model performance on NVIDIA Jetson Xavier NX board shows a desirable balance between the inference time and the memory cost. We also show qualitatively the model robustness and efficiency across the diverse complex scenarios of these datasets. We hope this work facilitates the advancement of the UAV-based perception in such crucial industrial applications.

Details

IROS Conference 2023 Conference Paper

Pseudo-Stereo++: Cycled Generative Pseudo-Stereo for Monocular 3D Object Detection in Autonomous Driving

Ahmed Elhagry
Hang Dai
Abdulmotaleb El Saddik

Recently, the feature-level generation has demonstrated the effectiveness of pseudo-stereo synthesis in Monocular 3D Detection (M3D). In this paper, we aim to further bridge the gap between the stereo and the monocular 3D object detectors in autonomous driving through direct image-level pseudo-stereo generation. We propose a novel Cycled Generative Pseudo-Stereo (CGPS) architecture to generate the right-view image from the left-view for constructing a pseudo-stereo pair to stereo 3D object detectors while maintaining the natural of M3D with the left-view image as the only input. Moreover, we use a triplet consistency loss to focus on the detected objects in the pseudo-stereo generation. Besides, we demonstrate that the proposed CGPS is an ad-hoc module to adapt top stereo 3D object detectors into monocular 3D object detectors. The proposed framework with CGPS achieves 74. 80%, 55. 28%, and 46. 70% 3DAP for easy, moderate, and hard difficulty levels in monocular 3D detection on the KITTI benchmark with comparable performance to the stereo 3D object detectors but using a monocular image as the only input. Till the submission, the proposed M3D framework ranks 1 st with dramatic improvements against the existing monocular 3D detectors on the KITTI benchmark.

Details

NeurIPS Conference 2023 Conference Paper

Weakly Supervised 3D Open-vocabulary Segmentation

Kunhao Liu
Fangneng Zhan
Jiahui Zhang
Muyu Xu
Yingchen Yu
Abdulmotaleb El Saddik
Christian Theobalt
Eric Xing

Open-vocabulary segmentation of 3D scenes is a fundamental function of human perception and thus a crucial objective in computer vision research. However, this task is heavily impeded by the lack of large-scale and diverse 3D open-vocabulary segmentation datasets for training robust and generalizable models. Distilling knowledge from pre-trained 2D open-vocabulary segmentation models helps but it compromises the open-vocabulary feature as the 2D models are mostly finetuned with close-vocabulary datasets. We tackle the challenges in 3D open-vocabulary segmentation by exploiting pre-trained foundation models CLIP and DINO in a weakly supervised manner. Specifically, given only the open-vocabulary text descriptions of the objects in a scene, we distill the open-vocabulary multimodal knowledge and object reasoning capability of CLIP and DINO into a neural radiance field (NeRF), which effectively lifts 2D features into view-consistent 3D segmentation. A notable aspect of our approach is that it does not require any manual segmentation annotations for either the foundation models or the distillation process. Extensive experiments show that our method even outperforms fully supervised models trained with segmentation annotations in certain scenes, suggesting that 3D open-vocabulary segmentation can be effectively learned from 2D images and text-image pairs. Code is available at https: //github. com/Kunhao-Liu/3D-OVS.

PDF Details

TIST Journal 2015 Journal Article

A Combined Approach Toward Consistent Reconstructions of Indoor Spaces Based on 6D RGB-D Odometry and KinectFusion

Nadia Figueroa
Haiwei Dong
Abdulmotaleb El Saddik

We propose a 6D RGB-D odometry approach that finds the relative camera pose between consecutive RGB-D frames by keypoint extraction and feature matching both on the RGB and depth image planes. Furthermore, we feed the estimated pose to the highly accurate KinectFusion algorithm, which uses a fast ICP (Iterative Closest Point) to fine-tune the frame-to-frame relative pose and fuse the depth data into a global implicit surface. We evaluate our method on a publicly available RGB-D SLAM benchmark dataset by Sturm et al. The experimental results show that our proposed reconstruction method solely based on visual odometry and KinectFusion outperforms the state-of-the-art RGB-D SLAM system accuracy. Moreover, our algorithm outputs a ready-to-use polygon mesh (highly suitable for creating 3D virtual worlds) without any postprocessing steps.

Details DOI

IROS Conference 2014 Conference Paper

Towards consistent reconstructions of indoor spaces based on 6D RGB-D odometry and KinectFusion

Haiwei Dong 0001
Nadia Figueroa
Abdulmotaleb El Saddik

We focus on generating consistent reconstructions of indoor spaces from a freely moving handheld RGB-D sensor, with the aim of creating virtual models that can be used for measuring and remodeling. We propose a novel 6D RGBD odometry approach that finds the relative camera pose between consecutive RGB-D frames by keypoint extraction and feature matching both on the RGB and depth image planes. Furthermore, we feed the estimated pose to the highly accurate KinectFusion algorithm, which uses a fast ICP (Iterative-Closest-Point) to fine-tune the frame-to-frame relative pose and fuse the Depth data into a global implicit surface. We evaluate our method on a publicly available RGB-D SLAM benchmark dataset by Sturm et al. The experimental results show that our proposed reconstruction method solely based on visual odometry and KinectFusion outperforms the state-of-the-art RGB-D SLAM system accuracy. Our algorithm outputs a ready-to-use polygon mesh (highly suitable for creating 3D virtual worlds) without any post-processing steps.

Details

IROS Conference 2013 Conference Paper

"Anti-fatigue" control for over-actuated bionic arm with muscle force constraints

Haiwei Dong 0001
Setareh Yazdkhasti
Nadia Figueroa
Abdulmotaleb El Saddik

In this paper, we propose an “anti-fatigue” control method for bionic actuated systems. Specifically, the proposed method is illustrated on an over-actuated bionic arm. Our control method consists of two steps. In the first step, a set of linear equations is derived by connecting the acceleration description in both joint and muscle space. The pseudo inverse solution to these equations provides an initial optimal muscle force distribution. As a second step, we derive a gradient direction for muscle force redistribution. This allows the muscles to satisfy force constraints and generate an even distribution of forces throughout all the muscles (i. e. towards "anti-fatigue"). The overall proposed method is tested for a bending-stretching movement. We used two models (bionic arm with 6 and 10 muscles) to verify the method. The force distribution analysis verifies the “anti-fatigue” property of the computed muscle force. The efficiency comparison shows that the computational time does not increase significantly with the increase of muscle number. The tracking error statistics of the two models show the validity of the method.

Details