Author name cluster

Xiao Ma

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

24 papers

2 author rows

AIIM Journal 2025 Journal Article

BDFormer: Boundary-aware dual-decoder transformer for skin lesion segmentation

Zexuan Ji
Yuxuan Ye
Xiao Ma

Segmenting skin lesions from dermatoscopic images is crucial for improving the quantitative analysis of skin cancer. However, automatic segmentation of skin lesions remains a challenging task due to the presence of unclear boundaries, artifacts, and obstacles such as hair and veins, all of which complicate the segmentation process. Transformers have demonstrated superior capabilities in capturing long-range dependencies through self-attention mechanisms and are gradually replacing CNNs in this domain. However, one of their primary limitations is the inability to effectively capture local details, which is crucial for handling unclear boundaries and significantly affects segmentation accuracy. To address this issue, we propose a novel boundary-aware dual-decoder transformer that employs a single encoder and dual-decoder framework for both skin lesion segmentation and dilated boundary segmentation. Within this model, we introduce a shifted window cross-attention block to build the dual-decoder structure and apply multi-task distillation to enable efficient interaction of inter-task information. Additionally, we propose a multi-scale aggregation strategy to refine the extracted features, ensuring optimal predictions. To further enhance boundary details, we incorporate a dilated boundary loss function, which expands the single-pixel boundary mask into planar information. We also introduce a task-wise consistency loss to promote consistency across tasks. Our method is evaluated on three datasets: ISIC2018, ISIC2017, and PH 2, yielding promising results with excellent performance compared to state-of-the-art models. The code is available at https: //github. com/Yuxuan-Ye/BDFormer.

Details DOI

NeurIPS Conference 2025 Conference Paper

BridgeVLA: Input-Output Alignment for Efficient 3D Manipulation Learning with Vision-Language Models

Peiyan Li
Yixiang Chen
Hongtao Wu
Xiao Ma
Xiangnan Wu
Yan Huang
Liang Wang
Tao Kong

Recently, leveraging pre-trained vision-language models (VLMs) for building vision-language-action (VLA) models has emerged as a promising approach to effective robot manipulation learning. However, only few methods incorporate 3D signals into VLMs for action prediction, and they do not fully leverage the spatial structure inherent in 3D data, leading to low data efficiency. In this paper, we introduce a new paradigm for constructing 3D VLAs. Specifically, we first pre-train the VLM backbone to take 2D images as input and produce 2D heatmaps as output. Using this pre-trained VLM as the backbone, we then fine-tune the entire VLA model while maintaining alignment between inputs and outputs by: (1) projecting raw point cloud inputs into multi-view images, and (2) predicting heatmaps before generating the final action. Extensive experiments show that the resulting model, BridgeVLA, can learn 3D manipulation both efficiently and effectively. BridgeVLA outperforms state-of-the-art baselines across three simulation benchmarks. In RLBench, it improves the average success rate from 81. 4\% to 88. 2\%. In COLOSSEUM, it demonstrates significantly better performance in challenging generalization settings, boosting the average success rate from 56. 7\% to 64. 0\%. In GemBench, it surpasses all the comparing baseline methods in terms of average success rate. In real-robot experiments, BridgeVLA outperforms a state-of-the-art baseline method by 32\% on average. It generalizes robustly in multiple out-of-distribution settings, including visual disturbances and unseen instructions. Remarkably, it is able to achieve a success rate of 95. 4\% on 10+ tasks with only 3 trajectories per task, while other VLA methods such as $\pi_{0}$ fail completely. Project Website: https: //bridgevla. github. io/.

PDF Details

NeurIPS Conference 2025 Conference Paper

Chain-of-Action: Trajectory Autoregressive Modeling for Robotic Manipulation

Wenbo Zhang
Tianrun Hu
Hanbo Zhang
Yanyuan Qiao
Yuchu Qin
Yang Li
Jiajun Liu
Tao Kong

We present Chain-of-Action (CoA), a novel visuomotor policy paradigm built upon Trajectory Autoregressive Modeling. Unlike conventional approaches that predict next step action(s) forward, CoA generates an entire trajectory by explicit backward reasoning with task-specific goals through an action-level Chain-of-Thought (CoT) process. This process is unified within a single autoregressive structure: (1) the first token corresponds to a stable keyframe action that encodes the task-specific goals; and (2) subsequent action tokens are generated autoregressively, conditioned on the initial keyframe and previously predicted actions. This backward action reasoning enforces a global-to-local structure, allowing each local action to be tightly constrained by the final goal. To further realize the action reasoning structure, CoA incorporates four complementary designs: continuous action token representation; dynamic stopping for variable-length trajectory generation; reverse temporal ensemble; and multi-token prediction to balance action chunk modeling with global structure. As a result, CoA gives strong spatial generalization capabilities while preserving the flexibility and simplicity of a visuomotor policy. Empirically, we observe that CoA outperforms representative imitation learning algorithms such as ACT and Diffusion Policy across 60 RLBench tasks and 8 real-world tasks.

PDF Details

ICRA Conference 2025 Conference Paper

Effective Tuning Strategies for Generalist Robot Manipulation Policies

Wenbo Zhang 0009
Yang Li
Yanyuan Qiao
Siyuan Huang 0004
Jiajun Liu
Feras Dayoub
Xiao Ma
Lingqiao Liu

Generalist robot manipulation policies (GMPs) have the potential to generalize across a wide range of tasks, devices, and environments. However, existing policies continue to struggle with out-of-distribution scenarios due to the inherent difficulty of collecting sufficient action data to cover extensively diverse domains. While fine-tuning offers a practical way to quickly adapt a GMPs to novel domains and tasks with limited samples, we observe that the performance of the resulting GMPs differs significantly with respect to the design choices of fine-tuning strategies. In this work, we first conduct an indepth empirical study to investigate the effect of key factors in GMPs fine-tuning strategies, covering the action space, policy head, supervision signal and the choice of tunable parameters, where 2, 500 rollouts are evaluated for a single configuration. We systematically discuss and summarize our findings and identify the key design choices, which we believe give a practical guideline for GMPs fine-tuning. We observe that in a lowdata regime, with carefully chosen fine-tuning strategies, a GMPs significantly outperforms the state-of-the-art imitation learning algorithms. The results presented in this work establish a new baseline for future studies on fine-tuned GMPs.

Details

NeurIPS Conference 2025 Conference Paper

Flow-Based Policy for Online Reinforcement Learning

Lei Lyu
Yunfei Li
Yu Luo
Fuchun Sun
Tao Kong
Jiafeng Xu
Xiao Ma

We present $\textbf{FlowRL}$, a novel framework for online reinforcement learning that integrates flow-based policy representation with Wasserstein-2-regularized optimization. We argue that in addition to training signals, enhancing the expressiveness of the policy class is crucial for the performance gains in RL. Flow-based generative models offer such potential, excelling at capturing complex, multimodal action distributions. However, their direct application in online RL is challenging due to a fundamental objective mismatch: standard flow training optimizes for static data imitation, while RL requires value-based policy optimization through a dynamic buffer, leading to difficult optimization landscapes. FlowRL first models policies via a state-dependent velocity field, generating actions through deterministic ODE integration from noise. We derive a constrained policy search objective that jointly maximizes Q through the flow polciy while bounding the Wasserstein-2 distance to a behavior-optimal policy implicitly derived from the replay buffer. This formulation effectively aligns the flow optimization with the RL objective, enabling efficient and value-aware policy learning despite the complexity of the policy class. Empirical evaluations on DMControl and Humanoidbench demonstrate that FlowRL achieves competitive performance in online reinforcement learning benchmarks.

PDF Details

NeurIPS Conference 2025 Conference Paper

Human-assisted Robotic Policy Refinement via Action Preference Optimization

Wenke Xia
Yichu Yang
Hongtao Wu
Xiao Ma
Tao Kong
Di Hu

Establishing a reliable and iteratively refined robotic system is essential for deploying real-world applications. While Vision-Language-Action (VLA) models are widely recognized as the foundation model for such robotic deployment, their reliance on offline expert demonstrations critically limits their capacity for post-deployment refinement. To mitigate this limitation, we introduce Action Preference Optimization (APO), a method designed to refine VLA models by human-assisted preference alignment gathered through interaction with environments. This method begins with a human-robot collaboration framework for reliable failure correction and interaction trajectory collection through human intervention. However, directly leveraging these interaction trajectories for preference optimization is non-trivial due to the challenges of irreversible robotic actions and token distribution mismatch. To solve this, APO proposes an adaptive reweighting algorithm with binary desirability signals derived from interaction, empowering VLA models effectively suppress failure-prone actions while enhancing corrective action adaptation. Ultimately, APO equips VLA models with the crucial capability to learn from failure, paving the way for their iterative refinement and reliable deployment in dynamic environments. The experiments conducted in simulation and real-world scenarios prove superior generalization and robustness of our human-assisted framework across a variety of manipulation tasks. We believe this work could bring insights for efficient and stable optimization of VLA models through human-robot collaboration. The code and dataset are released at https: //github. com/GeWu-Lab/Action-Preference-Optimization.

PDF Details

AAAI Conference 2024 Conference Paper

Collaborative Weakly Supervised Video Correlation Learning for Procedure-Aware Instructional Video Analysis

Tianyao He
Huabin Liu
Yuxi Li
Xiao Ma
Cheng Zhong
Yang Zhang
Weiyao Lin

Video Correlation Learning (VCL), which aims to analyze the relationships between videos, has been widely studied and applied in various general video tasks. However, applying VCL to instructional videos is still quite challenging due to their intrinsic procedural temporal structure. Specifically, procedural knowledge is critical for accurate correlation analyses on instructional videos. Nevertheless, current procedure-learning methods heavily rely on step-level annotations, which are costly and not scalable. To address this problem, we introduce a weakly supervised framework called Collaborative Procedure Alignment (CPA) for procedure-aware correlation learning on instructional videos. Our framework comprises two core modules: collaborative step mining and frame-to-step alignment. The collaborative step mining module enables simultaneous and consistent step segmentation for paired videos, leveraging the semantic and temporal similarity between frames. Based on the identified steps, the frame-to-step alignment module performs alignment between the frames and steps across videos. The alignment result serves as a measurement of the correlation distance between two videos. We instantiate our framework in two distinct instructional video tasks: sequence verification and action quality assessment. Extensive experiments validate the effectiveness of our approach in providing accurate and interpretable correlation analyses for instructional videos.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

MECD: Unlocking Multi-Event Causal Discovery in Video Reasoning

Tieyuan Chen
Huabin Liu
Tianyao He
Yihang Chen
Chaofan Gan
Xiao Ma
Cheng Zhong
Yang Zhang

Video causal reasoning aims to achieve a high-level understanding of video content from a causal perspective. However, current video reasoning tasks are limited in scope, primarily executed in a question-answering paradigm and focusing on short videos containing only a single event and simple causal relationships, lacking comprehensive and structured causality analysis for videos with multiple events. To fill this gap, we introduce a new task and dataset, Multi-Event Causal Discovery (MECD). It aims to uncover the causal relationships between events distributed chronologically across long videos. Given visual segments and textual descriptions of events, MECD requires identifying the causal associations between these events to derive a comprehensive, structured event-level video causal diagram explaining why and how the final result event occurred. To address MECD, we devise a novel framework inspired by the Granger Causality method, using an efficient mask-based event prediction model to perform an Event Granger Test, which estimates causality by comparing the predicted result event when premise events are masked versus unmasked. Furthermore, we integrate causal inference techniques such as front-door adjustment and counterfactual inference to address challenges in MECD like causality confounding and illusory causality. Experiments validate the effectiveness of our framework in providing causal relationships in multi-event videos, outperforming GPT-4o and VideoLLaVA by 5. 7% and 4. 1%, respectively.

PDF Details DOI

JBHI Journal 2024 Journal Article

MSGM: An Advanced Deep Multi-Size Guiding Matching Network for Whole Slide Histopathology Images Addressing Staining Variation and Low Visibility Challenges

Xiaoxiao Li
Zhengxiong Li
Taobo Hu
Mengping Long
Xiao Ma
Jin Huang
Yiqiang Liu
Yaxiaer Yalikun

Matching whole slide histopathology images to provide comprehensive information on homologous tissues is beneficial for cancer diagnosis. However, the challenge arises with the Giga-pixel whole slide images (WSIs) when aiming for high-accuracy matching. Learning-based methods are difficult to generalize well with large-size WSIs, necessitating the integration of traditional matching methods to enhance accuracy as the size increases. In this paper, we propose a multi-size guiding matching method applicable high-accuracy requirements. Specifically, we design learning multiscale texture to train deep descriptors, called TDescNet, that trains 64 × 64 × 256 and 256 × 256 × 128 size convolution layer as C64 and C256 descriptors to overcome staining variation and low visibility challenges. Furthermore, we develop the 3D-ring descriptor using sparse keypoints to support the description of large-size WSIs. Finally, we employ C64, C256, and 3D-ring descriptors to progressively guide refined local matching, utilizing geometric consistency to identify correct matching results. Experiments show that when matching WSIs of size 4096 × 4096 pixels, our average matching error is 123. 48 μm and the success rate is 93. 02 $\%$ in 43 cases. Notably, our method achieves an average improvement of 65. 52 μm in matching accuracy compared to recent state-of-the-art methods, with enhancements ranging from 36. 27 μm to 131. 66 μm. Therefore, we achieve high-fidelity whole-slice image matching, and overcome staining variation and low visibility challenges, enabling assistance in comprehensive cancer diagnosis through matched WSIs.

Details DOI

AAAI Conference 2024 Conference Paper

Responding to the Call: Exploring Automatic Music Composition Using a Knowledge-Enhanced Model

Zhejing Hu
Yan Liu
Gong Chen
Xiao Ma
Shenghua Zhong
Qianwen Luo

Call-and-response is a musical technique that enriches the creativity of music, crafting coherent musical ideas that mirror the back-and-forth nature of human dialogue with distinct musical characteristics. Although this technique is integral to numerous musical compositions, it remains largely uncharted in automatic music composition. To enhance the creativity of machine-composed music, we first introduce the Call-Response Dataset (CRD) containing 19,155 annotated musical pairs and crafted comprehensive objective evaluation metrics for musical assessment. Then, we design a knowledge-enhanced learning-based method to bridge the gap between human and machine creativity. Specifically, we train the composition module using the call-response pairs, supplementing it with musical knowledge in terms of rhythm, melody, and harmony. Our experimental results underscore that our proposed model adeptly produces a wide variety of creative responses for various musical calls.

PDF Details DOI

ICRA Conference 2024 Conference Paper

Weighting Online Decision Transformer with Episodic Memory for Offline-to-Online Reinforcement Learning

Xiao Ma
Wu-Jun Li

Offline reinforcement learning (RL) has been shown to be successfully modeled as a sequence modeling problem, drawing inspiration from the success of Transformers. Offline RL is often limited by the quality of the offline dataset, so offline-to-online RL is a more realistic setting. Online decision transformer (ODT) is an effective and representative sequence modeling-based offline-to-online RL method. Despite its effectiveness, ODT still suffers from the sample inefficiency problem during the online fine-tuning phase. This sample inefficiency problem arises because the agent treats all state-action pairs in the replay buffer equally when trying to learn from the replay buffer. In this paper, we propose a simple yet effective method, called weighting online decision transformer with episodic memory (WODTEM), to improve sample efficiency. We first attempt to introduce an episodic memory (EM) mechanism into the sequence modeling-based RL methods. By utilizing the EM mechanism, we propose a novel training objective with a weighting function, based on ODT, to improve sample efficiency. Experimental results on multiple tasks show that WODTEM can improve sample efficiency.

Details

YNIMG Journal 2024 Journal Article

Wireless optically pumped magnetometer MEG

Hao Cheng
Kaiyan He
Congcong Li
Xiao Ma
Fufu Zheng
Wei Xu
Pan Liao
Rui Yang

The current magnetoencephalography (MEG) systems, which rely on cables for control and signal transmission, do not fully realize the potential of wearable optically pumped magnetometers (OPM). This study presents a significant advancement in wireless OPM-MEG by reducing magnetization in the electronics and developing a tailored wireless communication protocol. Our protocol effectively eliminates electromagnetic interference, particularly in the critical frequency bands of MEG signals, and accurately synchronizes the acquisition and stimulation channels with the host computer's clock. We have successfully achieved single-channel wireless OPM-MEG measurement and demonstrated its reliability by replicating three well-established experiments: The alpha rhythm, auditory evoked field, and steady-state visual evoked field in the human brain. Our prototype wireless OPM-MEG system not only streamlines the measurement process but also represents a major step forward in the development of wearable OPM-MEG applications in both neuroscience and clinical research.

Details DOI

NeurIPS Conference 2023 Conference Paper

Efficient Diffusion Policies For Offline Reinforcement Learning

Bingyi Kang
Xiao Ma
Chao Du
Tianyu Pang
Shuicheng Yan

Offline reinforcement learning (RL) aims to learn optimal policies from offline datasets, where the parameterization of policies is crucial but often overlooked. Recently, Diffsuion-QL significantly boosts the performance of offline RL by representing a policy with a diffusion model, whose success relies on a parametrized Markov Chain with hundreds of steps for sampling. However, Diffusion-QL suffers from two critical limitations. 1) It is computationally inefficient to forward and backward through the whole Markov chain during training. 2) It is incompatible with maximum likelihood-based RL algorithms (e. g. , policy gradient methods) as the likelihood of diffusion models is intractable. Therefore, we propose efficient diffusion policy (EDP) to overcome these two challenges. EDP approximately constructs actions from corrupted ones at training to avoid running the sampling chain. We conduct extensive experiments on the D4RL benchmark. The results show that EDP can reduce the diffusion policy training time from 5 days to 5 hours on gym-locomotion tasks. Moreover, we show that EDP is compatible with various offline RL algorithms (TD3, CRR, and IQL) and achieves new state-of-the-art on D4RL by large margins over previous methods.

PDF Details

AAMAS Conference 2023 Conference Paper

Grey-box Adversarial Attack on Communication in Multi-agent Reinforcement Learning

Xiao Ma
Wu-Jun Li

Although research on communication in multi-agent reinforcement learning (MARL) has achieved some progress, the vulnerability of the communication mechanism in MARL caused by adversarial communication messages generated by malicious agents has not been well investigated. Existing works about adversarial communication messages in MARL focus on the black-box scenario where the attacker cannot access any model information about the multiagent system (MAS). But a more practical setting is the grey-box scenario where the attacker can access the model information about its controlled agent. To the best of our knowledge, there has not been any work investigating grey-box attacks on communication in MARL. In this paper, we propose the first grey-box attack method on communication in MARL, which is called victim-simulation based adversarial attack (VSA). At each timestep, the attacker simulates a victim attacked by other regular agents’ communication messages and generates adversarial perturbations on its received communication messages. The aggregation of these perturbations is sent by the attacker to the regular agents through communication messages, which will induce non-optimal actions of the regular agents. Experimental results show that VSA can effectively degrade the performance of the MAS on Predator-Prey. The findings in this paper will make researchers aware of the grey-box attack in MARL.

PDF

NeurIPS Conference 2023 Conference Paper

InsActor: Instruction-driven Physics-based Characters

Jiawei Ren
Mingyuan Zhang
Cunjun Yu
Xiao Ma
Liang Pan
Ziwei Liu

Generating animation of physics-based characters with intuitive control has long been a desirable task with numerous applications. However, generating physically simulated animations that reflect high-level human instructions remains a difficult problem due to the complexity of physical environments and the richness of human language. In this paper, we present $\textbf{InsActor}$, a principled generative framework that leverages recent advancements in diffusion-based human motion models to produce instruction-driven animations of physics-based characters. Our framework empowers InsActor to capture complex relationships between high-level human instructions and character motions by employing diffusion policies for flexibly conditioned motion planning. To overcome invalid states and infeasible state transitions in planned motions, InsActor discovers low-level skills and maps plans to latent skill sequences in a compact latent space. Extensive experiments demonstrate that InsActor achieves state-of-the-art results on various tasks, including instruction-driven motion generation and instruction-driven waypoint heading. Notably, the ability of InsActor to generate physically simulated animations using high-level human instructions makes it a valuable tool, particularly in executing long-horizon tasks with a rich set of instructions. Our project page is available at [jiawei-ren. github. io/projects/insactor/index. html](https: //jiawei-ren. github. io/projects/insactor/index. html)

PDF Details

JBHI Journal 2023 Journal Article

LAGAN: Lesion-Aware Generative Adversarial Networks for Edema Area Segmentation in SD-OCT Images

Yuhui Tao
Xiao Ma
Yizhe Zhang
Kun Huang
Zexuan Ji
Wen Fan
Songtao Yuan
Qiang Chen

Large volume of labeled data is a cornerstone for deep learning (DL) based segmentation methods. Medical images require domain experts to annotate, and full segmentation annotations of large volumes of medical data are difficult, if not impossible, to acquire in practice. Compared with full annotations, image-level labels are multiple orders of magnitude faster and easier to obtain. Image-level labels contain rich information that correlates with the underlying segmentation tasks and should be utilized in modeling segmentation problems. In this article, we aim to build a robust DL-based lesion segmentation model using only image-level labels (normal v. s. abnormal). Our method consists of three main steps: (1) training an image classifier with image-level labels; (2) utilizing a model visualization tool to generate an object heat map for each training sample according to the trained classifier; (3) based on the generated heat maps (as pseudo-annotations) and an adversarial learning framework, we construct and train an image generator for Edema Area Segmentation (EAS). We name the proposed method Lesion-Aware Generative Adversarial Networks (LAGAN) as it combines the merits of supervised learning (being lesion-aware) and adversarial training (for image generation). Additional technical treatments, such as the design of a multi-scale patch-based discriminator, further enhance the effectiveness of our proposed method. We validate the superior performance of LAGAN via comprehensive experiments on two publicly available datasets (i. e. , AI Challenger and RETOUCH).

Details DOI

NeurIPS Conference 2023 Conference Paper

Mutual Information Regularized Offline Reinforcement Learning

Xiao Ma
Bingyi Kang
Zhongwen Xu
Min Lin
Shuicheng Yan

The major challenge of offline RL is the distribution shift that appears when out-of-distribution actions are queried, which makes the policy improvement direction biased by extrapolation errors. Most existing methods address this problem by penalizing the policy or value for deviating from the behavior policy during policy improvement or evaluation. In this work, we propose a novel MISA framework to approach offline RL from the perspective of Mutual Information between States and Actions in the dataset by directly constraining the policy improvement direction. MISA constructs lower bounds of mutual information parameterized by the policy and Q-values. We show that optimizing this lower bound is equivalent to maximizing the likelihood of a one-step improved policy on the offline dataset. Hence, we constrain the policy improvement direction to lie in the data manifold. The resulting algorithm simultaneously augments the policy evaluation and improvement by adding mutual information regularizations. MISA is a general framework that unifies conservative Q-learning (CQL) and behavior regularization methods (e. g. , TD3+BC) as special cases. We introduce 3 different variants of MISA, and empirically demonstrate that tighter mutual information lower bound gives better offline RL performance. In addition, our extensive experiments show MISA significantly outperforms a wide range of baselines on various tasks of the D4RL benchmark, e. g. , achieving 742. 9 total points on gym-locomotion tasks. Our code is attached and will be released upon publication.

PDF Details

YNIMG Journal 2023 Journal Article

The early stage of adult ocular dominance plasticity revealed by near-infrared optical imaging of intrinsic signals

Wei Wu
Luwei Kang
Yueqin Liu
Xiao Ma
Xinxin Zhang
Yupeng Yang

Long term monocular deprivation is considered to be necessary for the induction of significant ocular dominance plasticity in the adult visual cortex. In this study, we subjected adult mice to monocular deprivation for various durations and screened for changes in ocular dominance using dual-wavelength intrinsic signal optical imaging. We found that short-term deprivation was sufficient to cause a shift in ocular dominance and that these early-stage changes were detected only by near-infrared illumination. In addition, single-unit recordings showed that these early-stage changes primarily occurred in deep cortical layers. This early-stage ocular dominance shift was abolished by the blockade of NMDA receptors. In summary, our findings reveal an early phase of adult ocular dominance plasticity and provide the dynamics of adult plasticity.

Details DOI

JBHI Journal 2022 Journal Article

LamNet: A Lesion Attention Maps-Guided Network for the Prediction of Choroidal Neovascularization Volume in SD-OCT Images

Yuhan Zhang
Xiao Ma
Mingchao Li
Zexuan Ji
Songtao Yuan
Qiang Chen

Choroidal neovascularization (CNV) volume prediction has an important clinical significance to predict the therapeutic effect and schedule the follow-up. In this paper, we propose a Lesion Attention Maps-Guided Network (LamNet) to automatically predict the CNV volume of next follow-up visit after therapy based on 3-dimentional spectral-domain optical coherence tomography (SD-OCT) images. In particular, the backbone of LamNet is a 3D convolutional neural network (3D-CNN). In order to guide the network to focus on the local CNV lesion regions, we use CNV attention maps generated by an attention map generator to produce the multi-scale local context features. Then, the multi-scale of both local and global feature maps are fused to achieve the high-precision CNV volume prediction. In addition, we also design a synergistic multi-task predictor, in which a trend-consistent loss ensures that the change trend of the predicted CNV volume is consistent with the real change trend of the CNV volume. The experiments include a total of 541 SD-OCT cubes from 68 patients with two types of CNV captured by two different SD-OCT devices. The results demonstrate that LamNet can provide the reliable and accurate CNV volume prediction, which would further assist the clinical diagnosis and design the treatment options.

Details DOI

YNIMG Journal 2022 Journal Article

Multimodal neuroimaging with optically pumped magnetometers: A simultaneous MEG-EEG-fNIRS acquisition system

Xingyu Ru
Kaiyan He
Bingjiang Lyu
Dongxu Li
Wei Xu
Wenyu Gu
Xiao Ma
Jiayi Liu

Multimodal neuroimaging plays an important role in neuroscience research. Integrated noninvasive neuroimaging modalities, such as magnetoencephalography (MEG), electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS), allow neural activity and related physiological processes in the brain to be precisely and comprehensively depicted, providing an effective and advanced platform to study brain function. Noncryogenic optically pumped magnetometer (OPM) MEG has high signal power due to its on-scalp sensor layout and enables more flexible configurations than traditional commercial superconducting MEG. Here, we integrate OPM-MEG with EEG and fNIRS to develop a multimodal neuroimaging system that can simultaneously measure brain electrophysiology and hemodynamics. We conducted a series of experiments to demonstrate the feasibility and robustness of our MEG-EEG-fNIRS acquisition system. The complementary neural and physiological signals simultaneously collected by our multimodal imaging system provide opportunities for a wide range of potential applications in neurovascular coupling, wearable neuroimaging, hyperscanning and brain-computer interfaces.

Details DOI

JBHI Journal 2022 Journal Article

Self-Supervised Sequence Recovery for Semi-Supervised Retinal Layer Segmentation

Jiadong Yang
Yuhui Tao
Qiuzhuo Xu
Yuhan Zhang
Xiao Ma
Songtao Yuan
Qiang Chen

Automated layer segmentation plays an important role for retinal disease diagnosis in optical coherence tomography (OCT) images. However, the severe retinal diseases result in the performance degeneration of automated layer segmentation approaches. In this paper, we present a robust semi-supervised layer segmentation network to relieve the model failures on abnormal retinas. We obtain the lesion features from the labeled images with disease-balanced distribution, and utilize the unlabeled images to supplement the layer structure information. Specifically, in our method, the cross-consistency training is utilized over the predictions of different decoders, and we enforce a consistency between different decoder predictions to improve the encoder's representation. Then, we propose a sequence prediction branch based on self-supervised manner, which is designed to predict the position of each jigsaw puzzle to obtain sensory perception of the retinal layer structure. To this task, a layer spatial pyramid pooling (LSPP) module is designed to extract multi-scale layer spatial features. Furthermore, we use the optical coherence tomography angiography (OCTA) to supplement the information damaged by diseases. The experimental results illustrate that our method achieves more robust results compared with current supervised segmentation methods. Meanwhile, advanced segmentation performance can be obtained compared with state-of-the-art semi-supervised segmentation methods.

Details DOI

NeurIPS Conference 2020 Conference Paper

Balanced Meta-Softmax for Long-Tailed Visual Recognition

Jiawei Ren
Cunjun Yu
shunan sheng
Xiao Ma
Haiyu Zhao
Shuai Yi
Hongsheng Li

Deep classifiers have achieved great success in visual recognition. However, real-world data is long-tailed by nature, leading to the mismatch between training and testing distributions. In this paper, we show that the Softmax function, though used in most classification tasks, gives a biased gradient estimation under the long-tailed setup. This paper presents Balanced Softmax, an elegant unbiased extension of Softmax, to accommodate the label distribution shift between training and testing. Theoretically, we derive the generalization bound for multiclass Softmax regression and show our loss minimizes the bound. In addition, we introduce Balanced Meta-Softmax, applying a complementary Meta Sampler to estimate the optimal class sample rate and further improve long-tailed learning. In our experiments, we demonstrate that Balanced Meta-Softmax outperforms state-of-the-art long-tailed classification solutions on both visual recognition and instance segmentation tasks.

PDF Details

JBHI Journal 2020 Journal Article

MS-CAM: Multi-Scale Class Activation Maps for Weakly-Supervised Segmentation of Geographic Atrophy Lesions in SD-OCT Images

Xiao Ma
Zexuan Ji
Sijie Niu
Theodore Leng
Daniel L. Rubin
Qiang Chen

As one of the most critical characteristics in advanced stage of non-exudative Age-related Macular Degeneration (AMD), Geographic Atrophy (GA) is one of the significant causes of sustained visual acuity loss. Automatic localization of retinal regions affected by GA is a fundamental step for clinical diagnosis. In this paper, we present a novel weakly supervised model for GA segmentation in Spectral-Domain Optical Coherence Tomography (SD-OCT) images. A novel Multi-Scale Class Activation Map (MS-CAM) is proposed to highlight the discriminatory significance regions in localization and detail descriptions. To extract available multi-scale features, we design a Scaling and UpSampling (SUS) module to balance the information content between features of different scales. To capture more discriminative features, an Attentional Fully Connected (AFC) module is proposed by introducing the attention mechanism into the fully connected operations to enhance the significant informative features and suppress less useful ones. Based on the location cues, the final GA region prediction is obtained by the projection segmentation of MS-CAM. The experimental results on two independent datasets demonstrate that the proposed weakly supervised model outperforms the conventional GA segmentation methods and can produce similar or superior accuracy comparing with fully supervised approaches. The source code has been released and is available on GitHub: https://github.com/jizexuan/Multi-Scale-Class-Activation-Map-Tensorflow.

Details DOI

AAAI Conference 2020 Conference Paper

Particle Filter Recurrent Neural Networks

Xiao Ma
Peter Karkus
David Hsu
Wee Sun Lee

Recurrent neural networks (RNNs) have been extraordinarily successful for prediction with sequential data. To tackle highly variable and multi-modal real-world data, we introduce Particle Filter Recurrent Neural Networks (PF-RNNs), a new RNN family that explicitly models uncertainty in its internal structure: while an RNN relies on a long, deterministic latent state vector, a PF-RNN maintains a latent state distribution, approximated as a set of particles. For effective learning, we provide a fully differentiable particle ﬁlter algorithm that updates the PF-RNN latent state distribution according to the Bayes rule. Experiments demonstrate that the proposed PF- RNNs outperform the corresponding standard gated RNNs on a synthetic robot localization dataset and 10 real-world sequence prediction datasets for text classiﬁcation, stock price prediction, etc.

PDF Details