Author name cluster

Jinwen Ma

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

17 papers

2 author rows

NeurIPS Conference 2025 Conference Paper

HybridNorm: Towards Stable and Efficient Transformer Training via Hybrid Normalization

Zhijian Zhuo
Yutao Zeng
Ya Wang
Sijun Zhang
Xiaoqing Li
Jian Yang
zhou Xun
Jinwen Ma

Transformers have become the de facto architecture for a wide range of machine learning tasks, particularly in large language models (LLMs). Despite their remarkable performance, many challenges remain in training deep transformer networks, especially regarding the position of the layer normalization. While Pre-Norm structures facilitate more stable training owing to their stronger identity path, they often lead to suboptimal performance compared to Post-Norm. In this paper, we propose HybridNorm, a simple yet effective hybrid normalization strategy that integrates the advantages of both Pre-Norm and Post-Norm. Specifically, HybridNorm employs QKV normalization within the attention mechanism and Post-Norm in the feed-forward network (FFN) of each transformer block. We provide both theoretical insights and empirical evidence to demonstrate that HybridNorm improves the gradient flow and the model robustness. Extensive experiments on large-scale transformer models, including both dense and sparse variants, show that HybridNorm consistently outperforms both Pre-Norm and Post-Norm approaches across multiple benchmarks. These findings highlight the potential of HybridNorm as a more stable and effective technique for improving the training and performance of deep transformer models. Code is available at https: //github. com/BryceZhuo/HybridNorm.

PDF Details

ICLR Conference 2025 Conference Paper

Polynomial Composition Activations: Unleashing the Dynamics of Large Language Models

Zhijian Zhuo 0001
Ya Wang 0002
Yutao Zeng
Xiaoqing Li
Xun Zhou
Jinwen Ma

Transformers have found extensive applications across various domains due to their powerful fitting capabilities. This success can be partially attributed to their inherent nonlinearity. Thus, in addition to the ReLU function employed in the original transformer architecture, researchers have explored alternative modules such as GeLU and SwishGLU to enhance nonlinearity and thereby augment representational capacity. In this paper, we propose a novel category of polynomial composition activations (PolyCom), designed to optimize the dynamics of transformers. Theoretically, we provide a comprehensive mathematical analysis of PolyCom, highlighting its enhanced expressivity and efficacy relative to other activation functions. Notably, we demonstrate that networks incorporating PolyCom achieve the **optimal approximation rate**, indicating that PolyCom networks require minimal parameters to approximate general smooth functions in Sobolev spaces. We conduct empirical experiments on the pre-training configurations of large language models (LLMs), including both dense and sparse architectures. By substituting conventional activation functions with PolyCom, we enable LLMs to capture higher-order interactions within the data, thus improving performance metrics in terms of accuracy and convergence rates. Extensive experimental results demonstrate the effectiveness of our method, showing substantial improvements over other activation functions. Code is available at https://github.com/BryceZhuo/PolyCom.

Details

ECAI Conference 2025 Conference Paper

Reframing Multimodal Complex Document Layout Understanding: A Layout-Aware Multi-Source Reasoning Decision Framework

Ran Chen 0002
Yuyang Zhou
Jingyang Deng
Zeren Zhang
Xuefei Tong
Jinwen Ma
Qinghui Shi
Qian Xu

Multimodal large language models (MLLMs) have achieved significant progress in document understanding. However, complex layout reasoning, characterized by concise answers and cross-page integration, remains a challenge. Unlike conventional semantics-oriented tasks, this task demands accurate visual perception of fine-grained structural elements and logical reasoning across multi-page documents. Existing approaches primarily focus on information extraction and semantic understanding, limiting the capacity of fine-tuned autoregressive models to capture short-answer reasoning signals and generalize to complex layout structures. To address this, we propose the Layout-Aware Multi-Source Reasoning Decision Framework (LAMRD), which reframes complex layout reasoning as a decision-making task over multi-source reasoning paths. In the reasoning path construction stage, LAMRD generates layout-aware reasoning paths by integrating internal visual cues and external knowledge from three complementary perspectives: Visual Structural Awareness (VSA), Logical Reasoning Paths (LRP), and External Knowledge Augmentation (EKA). In the reasoning path decision stage, we employ Group Relative Policy Optimization (GRPO) to train a decision model that produces the final answer based on these paths. We conduct comprehensive evaluations using Qwen2. 5-VL-7B-Instruct on the CEP-7K dataset, covering layout structure understanding, information extraction, and logical association. Experimental results demonstrate that LAMRD outperforms advanced MLLMs in accuracy, validating its effectiveness for complex document layout understanding.

Details

ECAI Conference 2025 Conference Paper

TopSUMseg: A Topology-Aware Swin Transformer-Mamba Framework for 3D Seismic Fault Image Segmentation

Ran Chen 0002
Jingyang Deng
Zeren Zhang
Ruohua Shi
Jinwen Ma

Seismic fault image segmentation is crucial for interpreting subsurface geological structures, supporting geologists in resource exploration and structural analysis. However, current deep learning models struggle with single-architecture limitations and the distinctive characteristics of seismic faults, which are distinguished by elongated structures with uneven spatial distributions. To address these challenges, we propose TopSUMseg, a novel Topology-Aware Swin Transformer-Mamba framework for 3D seismic fault image segmentation. Our framework combines Swin Transformer’s local feature extraction with Mamba’s efficient sequence modeling, and boosts 3D spatial modeling in Mamba with a newly designed Global-Local Attention module (GLA). Additionally, we design a Topology-Aware Structural Constraint (TASC) to align predictions with ground-truth structures in the feature space, promoting the modeling of complex fault geometries. Experiments on Thebe, the largest public seismic dataset, demonstrate that TopSUMseg achieves state-of-the-art performance with OIS and ODS scores of 0. 879 and 0. 875, respectively. Trained entirely from scratch, TopSUMseg nonetheless achieves superior performance compared to extensively pre-trained counterparts. In addition, TopSUMseg maintains a significantly lower parameter count while achieving a favorable trade-off between segmentation performance and time complexity, making it a practical and generalizable solution for real-world seismic fault interpretation.

Details

ECAI Conference 2024 Conference Paper

FltLM: An Intergrated Long-Context Large Language Model for Effective Context Filtering and Understanding

Jingyang Deng
Zhengyang Shen
Boyang Wang
Lixin Su
Suqi Cheng
Ying Nie 0006
Junfeng Wang 0009
Dawei Yin 0001

The development of Long-Context Large Language Models (LLMs) has markedly advanced natural language processing by facilitating the process of textual data across long documents and multiple corpora. However, Long-Context LLMs still face two critical challenges: The lost in the middle phenomenon, where crucial middle-context information is likely to be missed, and the distraction issue that the models lose focus due to overly extended contexts. To address these challenges, we propose the Context Filtering Language Model (FltLM), a novel integrated Long-Context LLM which enhances the ability of the model on multi-document question-answering (QA) tasks. Specifically, FltLM innovatively incorporates a context filter with a soft mask mechanism, identifying and dynamically excluding irrelevant content to concentrate on pertinent information for better comprehension and reasoning. Our approach not only mitigates these two challenges, but also enables the model to operate conveniently in a single forward pass. Experimental results demonstrate that FltLM significantly outperforms supervised fine-tuning and retrieval-based methods in complex QA scenarios, suggesting a promising solution for more accurate and reliable long-context natural language understanding applications.

Details

IJCAI Conference 2024 Conference Paper

Geometry-Guided Conditional Adaptation for Surrogate Models of Large-Scale 3D PDEs on Arbitrary Geometries

Jingyang Deng
Xingjian Li
Haoyi Xiong
Xiaoguang Hu
Jinwen Ma

Deep learning surrogate models aim to accelerate the solving of partial differential equations (PDEs) and have achieved certain promising results. Although several main-stream models through neural operator learning have been applied to delve into PDEs on varying geometries, they were designed to map the complex geometry to a latent uniform grid, which is still challenging to learn by the networks with general architectures. In this work, we rethink the critical factors of PDE solutions and propose a novel model-agnostic framework, called 3D Geometry-Guided Conditional adaptation (3D-GeoCA), for solving PDEs on arbitrary 3D geometries. Starting with a 3D point cloud geometry encoder, 3D-GeoCA can extract the essential and robust representations of any kind of geometric shapes, which conditionally guides the adaptation of hidden features in the surrogate model. We conduct experiments on two public computational fluid dynamics datasets, the Shape-Net Car and Ahmed-Body dataset, using several surrogate models as the backbones with various point cloud geometry encoders to simulate corresponding large-scale Reynolds Average Navier-Stokes equations. Equipped with 3D-GeoCA, these backbone models can reduce the L-2 error by a large margin. Moreover, this 3D-GeoCA is model-agnostic so that it can be applied to any surrogate model.

PDF Details DOI

ICLR Conference 2023 Conference Paper

Towards a Unified Theoretical Understanding of Non-contrastive Learning via Rank Differential Mechanism

Zhijian Zhuo 0001
Yifei Wang 0001
Jinwen Ma
Yisen Wang 0001

Recently, a variety of methods under the name of non-contrastive learning (like BYOL, SimSiam, SwAV, DINO) show that when equipped with some asymmetric architectural designs, aligning positive pairs alone is sufficient to attain good performance in self-supervised visual learning. Despite some understandings of some specific modules (like the predictor in BYOL), there is yet no unified theoretical understanding of how these seemingly different asymmetric designs can all avoid feature collapse, particularly considering methods that also work without the predictor (like DINO). In this work, we propose a unified theoretical understanding for existing variants of non-contrastive learning. Our theory named Rank Differential Mechanism (RDM) shows that all these asymmetric designs create a consistent rank difference in their dual-branch output features. This rank difference will provably lead to an improvement of effective dimensionality and alleviate either complete or dimensional feature collapse. Different from previous theories, our RDM theory is applicable to different asymmetric designs (with and without the predictor), and thus can serve as a unified understanding of existing non-contrastive learning methods. Besides, our RDM theory also provides practical guidelines for designing many new non-contrastive variants. We show that these variants indeed achieve comparable performance to existing methods on benchmark datasets, and some of them even outperform the baselines. Our code is available at \url{https://github.com/PKU-ML/Rank-Differential-Mechanism}.

Details

ICML Conference 2022 Conference Paper

PDO-s3DCNNs: Partial Differential Operator Based Steerable 3D CNNs

Zhengyang Shen
Tao Hong
Qi She
Jinwen Ma
Zhouchen Lin

Steerable models can provide very general and flexible equivariance by formulating equivariance requirements in the language of representation theory and feature fields, which has been recognized to be effective for many vision tasks. However, deriving steerable models for 3D rotations is much more difficult than that in the 2D case, due to more complicated mathematics of 3D rotations. In this work, we employ partial differential operators (PDOs) to model 3D filters, and derive general steerable 3D CNNs, which are called PDO-s3DCNNs. We prove that the equivariant filters are subject to linear constraints, which can be solved efficiently under various conditions. As far as we know, PDO-s3DCNNs are the most general steerable CNNs for 3D rotations, in the sense that they cover all common subgroups of SO(3) and their representations, while existing methods can only be applied to specific groups and representations. Extensive experiments show that our models can preserve equivariance well in the discrete domain, and outperform previous works on SHREC’17 retrieval and ISBI 2012 segmentation tasks with a low network complexity.

Details

IJCAI Conference 2021 Conference Paper

CIMON: Towards High-quality Hash Codes

Xiao Luo
Daqing Wu
Zeyu Ma
Chong Chen
Minghua Deng
Jinwen Ma
Zhongming Jin
Jianqiang Huang

Recently, hashing is widely used in approximate nearest neighbor search for its storage and computational efficiency. Most of the unsupervised hashing methods learn to map images into semantic similarity-preserving hash codes by constructing local semantic similarity structure from the pre-trained model as the guiding information, i. e. , treating each point pair similar if their distance is small in feature space. However, due to the inefficient representation ability of the pre-trained model, many false positives and negatives in local semantic similarity will be introduced and lead to error propagation during the hash code learning. Moreover, few of the methods consider the robustness of models, which will cause instability of hash codes to disturbance. In this paper, we propose a new method named Comprehensive sImilarity Mining and cOnsistency learNing (CIMON). First, we use global refinement and similarity statistical distribution to obtain reliable and smooth guidance. Second, both semantic and contrastive consistency learning are introduced to derive both disturb-invariant and discriminative hash codes. Extensive experiments on several benchmark datasets show that the proposed method outperforms a wide range of state-of-the-art methods in both retrieval performance and robustness.

PDF Details DOI

AAAI Conference 2021 Conference Paper

Continual Learning by Using Information of Each Class Holistically

Wenpeng Hu
Qi Qin
Mengyu Wang
Jinwen Ma
Bing Liu

Continual learning (CL) incrementally learns a sequence of tasks while solving the catastrophic forgetting (CF) problem. Existing methods mainly try to deal with CF directly. In this paper, we propose to avoid CF by considering the features of each class holistically rather than only the discriminative information for classifying the classes seen so far. This latter approach is prone to CF because the discriminative information for old classes may not be sufficiently discriminative for the new class to be learned. Consequently, in learning each new task, the network parameters for previous tasks have to be revised, which causes CF. With the holistic consideration, after adding new tasks, the system can still do well for previous tasks. The proposed technique is called Per-class Continual Learning (PCL). PCL has two key novelties. (1) It proposes a one-class learning based technique for CL, which considers features of each class holistically and represents a new approach to solving the CL problem. (2) It proposes a method to extract discriminative information after training to further improve the accuracy. Empirical evaluation shows that PCL markedly outperforms the state-of-the-art baselines for one or more classes per task. More tasks also result in more gains.

PDF Details

AAAI Conference 2021 Conference Paper

PDO-eS2CNNs: Partial Differential Operator Based Equivariant Spherical CNNs

Zhengyang Shen
Tiancheng Shen
Zhouchen Lin
Jinwen Ma

Spherical signals exist in many applications, e. g. , planetary data, LiDAR scans and digitalization of 3D objects, calling for models that can process spherical data effectively. It does not perform well when simply projecting spherical data into the 2D plane and then using planar convolution neural networks (CNNs), because of the distortion from projection and ineffective translation equivariance. Actually, good principles of designing spherical CNNs are avoiding distortions and converting the shift equivariance property in planar CNNs to rotation equivariance in the spherical domain. In this work, we use partial differential operators (PDOs) to design a spherical equivariant CNN, PDOeS2 CNN, which is exactly rotation equivariant in the continuous domain. We then discretize PDO-eS2 CNNs, and analyze the equivariance error resulted from discretization. This is the first time that the equivariance error is theoretically analyzed in the spherical domain. In experiments, PDOeS2 CNNs show greater parameter efficiency and outperform other spherical CNNs significantly on several tasks.

PDF Details

AAAI Conference 2021 Conference Paper

Predictive Adversarial Learning from Positive and Unlabeled Data

Wenpeng Hu
Ran Le
Bing Liu
Feng Ji
Jinwen Ma
Dongyan Zhao
Rui Yan

This paper studies learning from positive and unlabeled examples, known as PU learning. It proposes a novel PU learning method called Predictive Adversarial Networks (PAN) based on GAN (Generative Adversarial Networks). GAN learns a generator to generate data (e. g. , images) to fool a discriminator which tries to determine whether the generated data belong to a (positive) training class. PU learning can be casted as trying to identify (not generate) likely positive instances from the unlabeled set to fool a discriminator that determines whether the identified likely positive instances from the unlabeled set are indeed positive. However, directly applying GAN is problematic because GAN focuses on only the positive data. The resulting PU learning method will have high precision but low recall. We propose a new objective function based on KLdivergence. Evaluation using both image and text data shows that PAN outperforms state-of-the-art PU learning methods and also a direct adaptation of GAN for PU learning.

PDF Details

NeurIPS Conference 2020 Conference Paper

HRN: A Holistic Approach to One Class Learning

Wenpeng Hu
Mengyu Wang
Qi Qin
Jinwen Ma
Bing Liu

Existing neural network based one-class learning methods mainly use various forms of auto-encoders or GAN style adversarial training to learn a latent representation of the given one class of data. This paper proposes an entirely different approach based on a novel regularization, called holistic regularization (or H-regularization), which enables the system to consider the data holistically, not to produce a model that biases towards some features. Combined with a proposed 2-norm instance-level data normalization, we obtain an effective one-class learning method, called HRN. To our knowledge, the proposed regularization and the normalization method have not been reported before. Experimental evaluation using both benchmark image classification and traditional anomaly detection datasets show that HRN markedly outperforms the state-of-the-art existing deep/non-deep learning models.

PDF Details

AAAI Conference 2020 Conference Paper

Multi-Label Classification with Label Graph Superimposing

Ya Wang
Dongliang He
Fu Li
Xiang Long
Zhichao Zhou
Jinwen Ma
Shilei Wen

Images or videos always contain multiple objects or actions. Multi-label recognition has been witnessed to achieve pretty performance attribute to the rapid development of deep learning technologies. Recently, graph convolution network (GCN) is leveraged to boost the performance of multi-label recognition. However, what is the best way for label correlation modeling and how feature learning can be improved with label system awareness are still unclear. In this paper, we propose a label graph superimposing framework to improve the conventional GCN+CNN framework developed for multi-label recognition in the following two aspects. Firstly, we model the label correlations by superimposing label graph built from statistical co-occurrence information into the graph constructed from knowledge priors of labels, and then multilayer graph convolutions are applied on the ﬁnal superimposed graph for label embedding abstraction. Secondly, we propose to leverage embedding of the whole label system for better representation learning. In detail, lateral connections between GCN and CNN are added at shallow, middle and deep layers to inject information of label system into backbone CNN for label-awareness in the feature learning process. Extensive experiments are carried out on MS- COCO and Charades datasets, showing that our proposed solution can greatly improve the recognition performance and achieves new state-of-the-art recognition performance.

PDF Details

ICML Conference 2020 Conference Paper

PDO-eConvs: Partial Differential Operator Based Equivariant Convolutions

Zhengyang Shen
Lingshen He
Zhouchen Lin
Jinwen Ma

Recent research has shown that incorporating equivariance into neural network architectures is very helpful, and there have been some works investigating the equivariance of networks under group actions. However, as digital images and feature maps are on the discrete meshgrid, corresponding equivariance-preserving transformation groups are very limited. In this work, we deal with this issue from the connection between convolutions and partial differential operators (PDOs). In theory, assuming inputs to be smooth, we transform PDOs and propose a system which is equivariant to a much more general continuous group, the $n$-dimension Euclidean group. In implementation, we discretize the system using the numerical schemes of PDOs, deriving approximately equivariant convolutions (PDO-eConvs). Theoretically, the approximation error of PDO-eConvs is of the quadratic order. It is the first time that the error analysis is provided when the equivariance is approximate. Extensive experiments on rotated MNIST and natural image classification show that PDO-eConvs perform competitively yet use parameters much more efficiently. Particularly, compared with Wide ResNets, our methods result in better results using only 12. 6% parameters.

Details

IJCAI Conference 2019 Conference Paper

GSN: A Graph-Structured Network for Multi-Party Dialogues

Wenpeng Hu
Zhangming Chan
Bing Liu
Dongyan Zhao
Jinwen Ma
Rui Yan

Existing neural models for dialogue response generation assume that utterances are sequentially organized. However, many real-world dialogues involve multiple interlocutors (i. e. , multi-party dialogues), where the assumption does not hold as utterances from different interlocutors can occur ``in parallel. '' This paper generalizes existing sequence-based models to a Graph-Structured neural Network (GSN) for dialogue modeling. The core of GSN is a graph-based encoder that can model the information flow along the graph-structured dialogues (two-party sequential dialogues are a special case). Experimental results show that GSN significantly outperforms existing sequence-based models.

PDF Details

ICML Conference 2019 Conference Paper

The Anisotropic Noise in Stochastic Gradient Descent: Its Behavior of Escaping from Sharp Minima and Regularization Effects

Zhanxing Zhu
Jingfeng Wu
Bing Yu
Lei Wu
Jinwen Ma

Understanding the behavior of stochastic gradient descent (SGD) in the context of deep neural networks has raised lots of concerns recently. Along this line, we study a general form of gradient based optimization dynamics with unbiased noise, which unifies SGD and standard Langevin dynamics. Through investigating this general optimization dynamics, we analyze the behavior of SGD on escaping from minima and its regularization effects. A novel indicator is derived to characterize the efficiency of escaping from minima through measuring the alignment of noise covariance and the curvature of loss function. Based on this indicator, two conditions are established to show which type of noise structure is superior to isotropic noise in term of escaping efficiency. We further show that the anisotropic noise in SGD satisfies the two conditions, and thus helps to escape from sharp and poor minima effectively, towards more stable and flat minima that typically generalize well. We systematically design various experiments to verify the benefits of the anisotropic noise, compared with full gradient descent plus isotropic diffusion (i. e. Langevin dynamics).

Details