Author name cluster

Zhi Jin

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

27 papers

2 author rows

EAAI Journal 2026 Journal Article

A knowledge graph-driven generation framework for perceptual decomposition and serial logical reasoning with large language models

Yuhang Zheng
Xuan Zhang
KunPeng Du
Junda Li
Linyu Li
Shaobo Liu
Tong Li
Zhi Jin

Large Language Models have demonstrated outstanding performance across a wide range of tasks. However, they often suffer from hallucinations and outdated knowledge issues. Knowledge Graphs provide structured and reliable external knowledge to alleviate these issues effectively. Nevertheless, existing Knowledge Graph-based retrieval-augmented methods struggle to handle complex multi-hop reasoning tasks and are prone to error propagation during path nodes retrieval, while fine-tuning-based retrieval-augmented methods often suffer from poor generalization. To address these challenges, we propose Perceptual Decomposition and Serial logical Reasoning, a complexity aware reasoning method that combines retrieval and reasoning in a unified design. Specifically, we propose a complexity-aware decomposition method that assesses question complexity and decomposes the question into simpler sub-questions with logical relationships. This approach effectively addresses complex multi-hop questions and reduces redundancy. To avoid the issue of error propagation caused by incorrect retrieval of path nodes, we employ subgraph-level retrieval, which provides higher answer coverage, instead of path-based retrieval. Additionally, we design a serial logical reasoning strategy in which sub-questions are solved progressively based on their corresponding subgraphs and then integrated to generate the final answer, transforming complex global reasoning into manageable local reasoning steps Without fine-tuning, our method is evaluated on the WebQuestionsSP and ComplexWebQuestions datasets. On the WebQuestionsSP dataset, our method achieves an average improvement of over 6% in Hit and Macro-F1 scores compared to the existing state-of-the-art methods. Specifically, in multi-hop questions, the improvement in the Hit score exceeds 10%. The experimental results demonstrate its superiority, robustness, and practical value for complex reasoning tasks.

Details DOI

AAAI Conference 2026 Conference Paper

Gradient as Conditions: Rethinking HOG for All-in-one Image Restoration

Jiawei Wu
Zhifei Yang
Zhe Wang
Zhi Jin

All-in-one image restoration (AIR) aims to address diverse degradations within a unified model by leveraging informative degradation conditions to guide the restoration process. However, existing methods often rely on implicitly learned priors, which may entangle feature representations and hinder performance in complex or unseen scenarios. Histogram of Oriented Gradients (HOG) as a classical gradient representation, we observe that it has strong discriminative capability across diverse degradations, making it a powerful and interpretable prior for AIR. Based on this insight, we propose HOGformer, a Transformer-based model that integrates learnable HOG features for degradation-aware restoration. The core of HOGformer is a Dynamic HOG-aware Self-Attention (DHOGSA) mechanism, which adaptively models long-range spatial dependencies conditioned on degradation-specific cues encoded by HOG descriptors. To further adapt the heterogeneity of degradations in AIR, we propose a Dynamic Interaction Feed-Forward (DIFF) module that facilitates channel–spatial interactions, enabling robust feature transformation under diverse degradations. Besides, we propose a HOG loss to explicitly enhance structural fidelity and edge sharpness. Extensive experiments on a variety of benchmarks, including adverse weather and natural degradations, demonstrate that HOGformer achieves state-of-the-art performance and generalizes well to complex real-world scenarios.

PDF Details DOI

AAAI Conference 2026 Conference Paper

Large Language Model Unlearning for Source Code

Xue Jiang
Yihong Dong
Huangzhao Zhang
Tangxinyu Wang
Zheng Fang
Yingwei Ma
Rongyu Cao
Binhua Li

While Large Language Models (LLMs) excel at code generation, their inherent tendency toward verbatim memorization of training data introduces critical risks like copyright infringement, insecurity emission, and deprecated API utilization, etc. A straightforward yet promising defense is unlearning, i.e., erasing or down-weighting the offending snippets through post-training. However, we find its application to source code often tends to spill over, damaging the basic knowledge of programming languages learned by the LLM and degrading the overall capability. To ease this challenge, we propose PROD for precise source code unlearning. PROD surgically zeroes out the prediction probability of the prohibited tokens, and renormalizes the remaining distribution so that the generated code stays correct. By excising only the targeted snippets, PROD achieves precise forgetting without much degradation of the LLM's overall capability. To facilitate in-depth evaluation against PROD, we establish an unlearning benchmark consisting of three downstream tasks (i.e., unlearning of copyrighted code, insecure code, and deprecated APIs), and introduce Pareto Dominance Ratio (PDR) metric, which indicates both the forget quality and the LLM utility. Our comprehensive evaluation demonstrates that PROD achieves superior overall performance between forget quality and model utility compared to existing unlearning approaches across three downstream tasks, while consistently exhibiting improvements when applied to LLMs of varying series. PROD also exhibits superior robustness against adversarial attacks without generating or exposing the data to be forgotten. These results underscore that our approach not only successfully extends the application boundary of unlearning techniques to source code, but also holds significant implications for advancing reliable code generation.

PDF Details DOI

AAAI Conference 2026 Conference Paper

RESTL: Reinforcement Learning Guided by Multi-Aspect Rewards for Signal Temporal Logic Transformation

Yue Fang
Zhi Jin
Jie An
Hongshen Chen
Xiaohong Chen
Naijun Zhan

Signal Temporal Logic (STL) is a powerful formal language for specifying real-time specifications of Cyber-Physical Systems (CPS). Transforming specifications written in natural language into STL formulas automatically has attracted increasing attention. Existing rule-based methods depend heavily on rigid pattern matching and domain-specific knowledge, limiting their generalizability and scalability. Recently, Supervised Fine-Tuning (SFT) of large language models (LLMs) has been successfully applied to transform natural language into STL. However, the lack of fine-grained supervision on atomic proposition correctness, semantic fidelity, and formula readability often leads SFT-based methods to produce formulas misaligned with the intended meaning. To address these issues, we propose RESTL, a reinforcement learning (RL)-based framework for the transformation from natural language to STL. RESTL introduces multiple independently trained reward models that provide fine-grained, multi-faceted feedback from four perspectives, i.e., atomic proposition consistency, semantic alignment, formula succinctness, and symbol matching. These reward models are trained with a curriculum learning strategy to improve their feedback accuracy, and their outputs are aggregated into a unified signal that guides the optimization of the STL generator via Proximal Policy Optimization (PPO). Experimental results demonstrate that RESTL significantly outperforms state-of-the-art methods in both automatic metrics and human evaluations.

PDF Details DOI

AAAI Conference 2025 Conference Paper

A Comprehensive Evaluation on Event Reasoning of Large Language Models

Zhengwei Tao
Zhi Jin
Yifan Zhang
Xiancai Chen
Haiyan Zhao
Jia Li
Bin Liang
Chongyang Tao

Event reasoning is a fundamental ability that underlies many applications. It requires event schema knowledge to perform global reasoning and needs to deal with the diversity of the inter-event relations and the reasoning paradigms. The extent to which LLMs excel in event reasoning across various relations and reasoning paradigms has not been thoroughly investigated. Additionally, it is still unclear whether LLMs utilize event knowledge in the same way humans do. To mitigate this disparity, we comprehensively evaluate the abilities of event reasoning of LLMs on different relations, paradigms, and levels of abstraction. We introduce a novel benchmark EV2 for EValuation of EVent reasoning. EV2 consists of two levels of evaluation on schema and instance and is comprehensive in relations and reasoning paradigms. We conduct extensive experiments on EV2. We find that 1) LLMs have abilities to accomplish event reasoning but their performances are far from satisfactory. 2) There are imbalances of event reasoning abilities on different relations and paradigms. 3) LLMs have event schema knowledge, however, they're not aligned with humans on how to utilize the knowledge. Based on these findings, we guide the LLMs in utilizing the event schema knowledge as memory leading to improvements in event reasoning.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

Bidirectional Representations Augmented Autoregressive Biological Sequence Generation

Xiang Zhang
Jiaqi Wei
Zijie Qiu
Sheng Xu
Zhi Jin
Zhiqiang Gao
Nanqing Dong
Siqi Sun

Autoregressive (AR) models, common in sequence generation, are limited in many biological tasks like de novo peptide sequencing and protein modeling by their unidirectional nature, failing to capture crucial global bidirectional token dependencies. Non-Autoregressive (NAR) models offer holistic, bidirectional representations but face challenges with generative coherence and scalability. To transcend this, we propose a hybrid framework enhancing AR generation by dynamically integrating rich contextual information from non-autoregressive mechanisms. Our approach couples a shared input encoder with two decoders: a non-autoregressive one learning latent bidirectional biological features, and an AR decoder synthesizing the biological sequence by leveraging these bidirectional features. A novel cross-decoder attention module enables the AR decoder to iteratively query and integrate these bidirectional features, enriching its predictions. This synergy is cultivated via a tailored training strategy with importance annealing for balanced objectives and cross-decoder gradient blocking for stable, focused learning. Evaluations on a demanding 9-species benchmark of de novo peptide sequencing task show our model substantially surpasses AR and NAR baselines. It uniquely harmonizes AR stability with NAR contextual awareness, delivering robust, superior performance on diverse downstream data. This research advances biological sequence modeling techniques and contributes a novel architectural paradigm for augmenting AR models with enhanced bidirectional understanding for complex sequence generation. Our code is available on GitHub: https: //github. com/BEAM-Labs/denovo

PDF Details

NeurIPS Conference 2025 Conference Paper

Recursive Transformer: Boosting Reasoning Ability with State Stack

Kechi Zhang
Ge Li
Huangzhao Zhang
Yihong Dong
Jia Li
Jingjing Xu
Zhi Jin

The Transformer architecture has emerged as a landmark advancement within the broad field of artificial intelligence, effectively catalyzing the advent of large language models (LLMs). However, despite its remarkable capabilities and the substantial progress it has facilitated, the Transformer architecture still has some limitations. One such intrinsic limitation is its inability to effectively recognize regular expressions or deterministic context-free grammars. Standard Transformers lack an explicit mechanism for recursion and structured state transitions, which can hinder systematic generalization on nested and hierarchical patterns. Drawing inspiration from pushdown automata, which efficiently resolve deterministic context-free grammars using stacks, we equip layers with a differentiable stack and propose StackTrans with recursion to address the aforementioned issue within LLMs. Unlike previous approaches that modify the attention computation, StackTrans explicitly incorporates hidden state stacks between Transformer layers. This design maintains compatibility with existing frameworks like flash-attention. Specifically, our design features stack operations -- such as pushing and popping hidden states -- that are differentiable and can be learned in an end-to-end manner. Our comprehensive evaluation spans benchmarks for both Chomsky hierarchy and large-scale natural languages. Across these diverse tasks, StackTrans consistently outperforms standard Transformer models and other baselines. We have successfully scaled StackTrans up from 360M to 7B parameters. In particular, our from-scratch pretrained model StackTrans-360M outperforms several larger open-source LLMs with 2–3x more parameters, showcasing its superior efficiency and reasoning capability.

PDF Details

ICML Conference 2025 Conference Paper

Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing

Zijie Qiu
Jiaqi Wei
Xiang Zhang 0011
Sheng Xu
Kai Zou
Zhi Jin
Zhiqiang Gao
Nanqing Dong

De novo peptide sequencing is a critical task in proteomics. However, the performance of current deep learning-based methods is limited by the inherent complexity of mass spectrometry data and the heterogeneous distribution of noise signals, leading to data-specific biases. We present RankNovo, the first deep reranking framework that enhances de novo peptide sequencing by leveraging the complementary strengths of multiple sequencing models. RankNovo employs a list-wise reranking approach, modeling candidate peptides as multiple sequence alignments and utilizing axial attention to extract informative features across candidates. Additionally, we introduce two new metrics, PMD ( P eptide M ass D eviation) and RMD ( R esidual M ass D eviation), which offer delicate supervision by quantifying mass differences between peptides at both the sequence and residue levels. Extensive experiments demonstrate that RankNovo not only surpasses its base models used to generate training candidates for reranking pre-training, but also sets a new state-of-the-art benchmark. Moreover, RankNovo exhibits strong zero-shot generalization to unseen models—those whose generations were not exposed during training, highlighting its robustness and potential as a universal reranking framework for peptide sequencing. Our work presents a novel reranking strategy that fundamentally challenges existing single-model paradigms and advances the frontier of accurate de novo sequencing. Our source code is provided on GitHub.

Details

TAAS Journal 2024 Journal Article

A Game-Theoretical Self-Adaptation Framework for Securing Software-Intensive Systems

Nianyu Li
Mingyue Zhang
Jialong Li
Sridhar Adepu
Eunsuk Kang
Zhi Jin

Security attacks present unique challenges to the design of self-adaptation mechanism for software-intensive systems due to the adversarial nature of the environment. Game-theoretical approaches have been explored in security to model malicious behaviors and design reliable defense for the system in a mathematically grounded manner. However, modeling the system as a single player, as done in prior works, is insufficient for the system under partial compromise and for the design of fine-grained defensive policies where the rest of the system with autonomy can cooperate to mitigate the impact of attacks. To address such issues, we propose a new self-adaptation framework incorporating Bayesian game theory and model the defender (i.e., the system) at the granularity of components. Under security attacks, the architecture model of the system is automatically translated, by the proposed translation process with designed algorithms, into a multi-player Bayesian game. This representation allows each component to be modeled as an independent player, while security attacks are encoded as variant types for the components. By solving for pure equilibrium (i.e., adaptation response), the system’s optimal defensive strategy is dynamically computed, enhancing system resilience against security attacks by maximizing system utility. We validate the effectiveness of our framework through two sets of experiments using generic benchmark tasks tailored for the security domain. Additionally, we exemplify the practical application of our approach through a real-world implementation in the Secure Water Treatment System to demonstrate the applicability and potency in mitigating security risks.

Details DOI

AAAI Conference 2024 Conference Paper

ContraNovo: A Contrastive Learning Approach to Enhance De Novo Peptide Sequencing

Zhi Jin
Sheng Xu
Xiang Zhang
Tianze Ling
Nanqing Dong
Wanli Ouyang
Zhiqiang Gao
Cheng Chang

De novo peptide sequencing from mass spectrometry (MS) data is a critical task in proteomics research. Traditional de novo algorithms have encountered a bottleneck in accuracy due to the inherent complexity of proteomics data. While deep learning-based methods have shown progress, they reduce the problem to a translation task, potentially overlooking critical nuances between spectra and peptides. In our research, we present ContraNovo, a pioneering algorithm that leverages contrastive learning to extract the relationship between spectra and peptides and incorporates the mass information into peptide decoding, aiming to address these intricacies more efficiently. Through rigorous evaluations on two benchmark datasets, ContraNovo consistently outshines contemporary state-of-the-art solutions, underscoring its promising potential in enhancing de novo peptide sequencing.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

EvoCodeBench: An Evolving Code Generation Benchmark with Domain-Specific Evaluations

Jia Li
Ge Li
Xuanming Zhang
YunFei Zhao
Yihong Dong
Zhi Jin
Binhua Li
Fei Huang

How to evaluate Large Language Models (LLMs) in code generation remains an open question. Many benchmarks have been proposed, but they have two limitations, i. e. , data leakage and lack of domain-specific evaluation. The former hurts the fairness of benchmarks, and the latter hinders practitioners from selecting superior LLMs for specific programming domains. To address these two limitations, we propose a new benchmark - EvoCodeBench, which has the following advances: (1) Evolving data. EvoCodeBench will be dynamically updated every period (e. g. , 6 months) to avoid data leakage. This paper releases the first version - EvoCodeBench-2403, containing 275 samples from 25 repositories. (2) A domain taxonomy and domain labels. Based on the statistics of open-source communities, we design a programming domain taxonomy consisting of 10 popular domains. Based on the taxonomy, we annotate each sample in EvoCodeBench with a domain label. EvoCodeBench provides a broad platform for domain-specific evaluations. (3) Domain-specific evaluations. Besides the Pass@k, we compute the Domain-Specific Improvement (DSI) and define LLMs' comfort and strange domains. These evaluations help practitioners select superior LLMs in specific domains and discover the shortcomings of existing LLMs. Besides, EvoCodeBench is collected by a rigorous pipeline and aligns with real-world repositories in multiple aspects (e. g. , code distributions). We evaluate 8 popular LLMs (e. g. , gpt-4, DeepSeek Coder, StarCoder 2) on EvoCodeBench and summarize some insights. EvoCodeBench reveals the actual abilities of these LLMs in real-world repositories. For example, the highest Pass@1 of gpt-4 on EvoCodeBench-2403 is only 20. 74%. Besides, we evaluate LLMs in different domains and discover their comfort and strange domains. For example, gpt-4 performs best in most domains but falls behind others in the Internet domain. StarCoder 2-15B unexpectedly performs well in the Database domain and even outperforms 33B LLMs. We release EvoCodeBench, all prompts, and LLMs' completions for further community analysis.

PDF Details DOI

IJCAI Conference 2024 Conference Paper

FastScene: Text-Driven Fast Indoor 3D Scene Generation via Panoramic Gaussian Splatting

Yikun Ma
Dandan Zhan
Zhi Jin

Text-driven 3D indoor scene generation holds broad applications, ranging from gaming and smart homes to AR/VR applications. Fast and high-fidelity scene generation is paramount for ensuring user-friendly experiences. However, existing methods are characterized by lengthy generation processes or necessitate the intricate manual specification of motion parameters, which introduces inconvenience for users. Furthermore, these methods often rely on narrow-field viewpoint iterative generations, compromising global consistency and overall scene quality. To address these issues, we propose FastScene, a framework for fast and higher-quality 3D scene generation, while maintaining the scene consistency. Specifically, given a text prompt, we generate a panorama and estimate its depth, since the panorama encompasses information about the entire scene and exhibits explicit geometric constraints. To obtain high-quality novel views, we introduce the Coarse View Synthesis (CVS) and Progressive Novel View Inpainting (PNVI) strategies, ensuring both scene consistency and view quality. Subsequently, we utilize Multi-View Projection (MVP) to form perspective views, and apply 3D Gaussian Splatting (3DGS) for scene reconstruction. Comprehensive experiments demonstrate FastScene surpasses other methods in both generation speed and quality with better scene consistency. Notably, guided only by a text prompt, FastScene can generate a 3D scene within a mere 15 minutes, which is at least one hour faster than state-of-the-art methods, making it a paradigm for user-friendly scene generation.

PDF Details DOI

TAAS Journal 2024 Journal Article

Generative AI for Self-Adaptive Systems: State of the Art and Research Roadmap

Jialong Li
Mingyue Zhang
Nianyu Li
Danny Weyns
Zhi Jin
Kenji Tei

Self-adaptive systems (SASs) are designed to handle changes and uncertainties through a feedback loop with four core functionalities: monitoring, analyzing, planning, and execution. Recently, generative artificial intelligence (GenAI), especially the area of large language models, has shown impressive performance in data comprehension and logical reasoning. These capabilities are highly aligned with the functionalities required in SASs, suggesting a strong potential to employ GenAI to enhance SASs. However, the specific benefits and challenges of employing GenAI in SASs remain unclear. Yet, providing a comprehensive understanding of these benefits and challenges is complex due to several reasons: limited publications in the SAS field, the technological and application diversity within SASs, and the rapid evolution of GenAI technologies. To that end, this article aims to provide researchers and practitioners a comprehensive snapshot that outlines the potential benefits and challenges of employing GenAI’s within SAS. Specifically, we gather, filter, and analyze literature from four distinct research fields and organize them into two main categories to potential benefits: (i) enhancements to the autonomy of SASs centered around the specific functions of the MAPE-K feedback loop, and (ii) improvements in the interaction between humans and SASs within human-on-the-loop settings. From our study, we outline a research roadmap that highlights the challenges of integrating GenAI into SASs. The roadmap starts with outlining key research challenges that need to be tackled to exploit the potential for applying GenAI in the field of SAS. The roadmap concludes with a practical reflection, elaborating on current shortcomings of GenAI and proposing possible mitigation strategies. †

Details DOI

AAAI Conference 2024 Conference Paper

Hot or Cold? Adaptive Temperature Sampling for Code Generation with Large Language Models

Yuqi Zhu
Ge Li
YunFei Zhao
Jia Li
Zhi Jin
Hong Mei

Recently, Large Language Models (LLMs) have shown impressive abilities in code generation. However, existing LLMs' decoding strategies are designed for Natural Language (NL) generation, overlooking the differences between NL and programming languages (PL). Due to this oversight, a better decoding strategy for code generation remains an open question. In this paper, we conduct the first systematic study to explore a decoding strategy specialized in code generation. With an analysis of loss distributions of code tokens, we find that code tokens can be divided into two categories: challenging tokens that are difficult to predict and confident tokens that can be easily inferred. Among them, the challenging tokens mainly appear at the beginning of a code block. Inspired by the above findings, we propose a simple yet effective method: Adaptive Temperature (AdapT) sampling, which dynamically adjusts the temperature coefficient when decoding different tokens. We apply a larger temperature when sampling for challenging tokens, allowing LLMs to explore diverse choices. We employ a smaller temperature for confident tokens avoiding the influence of tail randomness noises. We apply AdapT sampling to LLMs with different sizes and conduct evaluations on two popular datasets. Results show that AdapT sampling significantly outperforms state-of-the-art decoding strategy.

PDF Details DOI

JBHI Journal 2024 Journal Article

MultiModRLBP: A Deep Learning Approach for Multi-Modal RNA-Small Molecule Ligand Binding Sites Prediction

Junkai Wang
Lijun Quan
Zhi Jin
Hongjie Wu
Xuhao Ma
Xuejiao Wang
Jingxin Xie
Deng Pan

This study aims to tackle the intricate challenge of predicting RNA-small molecule binding sites to explore the potential value in the field of RNA drug targets. To address this challenge, we propose the MultiModRLBP method, which integrates multi-modal features using deep learning algorithms. These features include 3D structural properties at the nucleotide base level of the RNA molecule, relational graphs based on overall RNA structure, and rich RNA semantic information. In our investigation, we gathered 851 interactions between RNA and small molecule ligand from the RNAglib dataset and RLBind training set. Unlike conventional training sets, this collection broadened its scope by including RNA complexes that have the same RNA sequence but change their respective binding sites due to structural differences or the presence of different ligands. This enhancement enables the MultiModRLBP model to more accurately capture subtle changes at the structural level, ultimately improving its ability to discern nuances among similar RNA conformations. Furthermore, we evaluated MultiModRLBP on two classic test sets, Test18 and Test3, highlighting its performance disparities on small molecules based on metal and non-metal ions. Additionally, we conducted a structural sensitivity analysis on specific complex categories, considering RNA instances with varying degrees of structural changes and whether they share the same ligands. The research results indicate that MultiModRLBP outperforms the current state-of-the-art methods on multiple classic test sets, particularly excelling in predicting binding sites for non-metal ions and instances where the binding sites are widely distributed along the sequence. MultiModRLBP also can be used as a potential tool when the RNA structure is perturbed or the RNA experimental tertiary structure is not available. Most importantly, MultiModRLBP exhibits the capability to distinguish binding characteristics of RNA that are structurally diverse yet exhibit sequence similarity. These advancements hold promise in reducing the costs associated with the development of RNA-targeted drugs.

Details DOI

NeurIPS Conference 2022 Conference Paper

Fine-Tuning Pre-Trained Language Models Effectively by Optimizing Subnetworks Adaptively

Haojie Zhang
Ge Li
Jia Li
Zhongjin Zhang
Yuqi Zhu
Zhi Jin

Large-scale pre-trained language models have achieved impressive results on a wide range of downstream tasks recently. However, fine-tuning an extremely large-scale pre-trained language model on limited target datasets is often plagued by overfitting and representation degradation. In this paper, we propose a Dynamic Parameter Selection (DPS) algorithm for the large-scale pre-trained models during fine-tuning, which adaptively selects a more promising subnetwork to perform staging updates based on gradients of back-propagation. Experiments on the GLUE benchmark show that DPS outperforms previous fine-tuning methods in terms of overall performance and stability, and consistently achieves better results with variable pre-trained language models. In addition, DPS brings a large magnitude of improvement in out-of-domain transferring experiments and low-resource scenarios, which shows that it can maintain stable general contextual features and reduce the representation collapse. We release our code at \url{https: //github. com/ZhangHaojie077/DPS}.

PDF Details

NeurIPS Conference 2022 Conference Paper

You Never Stop Dancing: Non-freezing Dance Generation via Bank-constrained Manifold Projection

Jiangxin Sun
Chunyu Wang
Huang Hu
Hanjiang Lai
Zhi Jin
Jian-Fang Hu

One of the most overlooked challenges in dance generation is that the auto-regressive frameworks are prone to freezing motions due to noise accumulation. In this paper, we present two modules that can be plugged into the existing models to enable them to generate non-freezing and high fidelity dances. Since the high-dimensional motion data are easily swamped by noise, we propose to learn a low-dimensional manifold representation by an auto-encoder with a bank of latent codes, which can be used to reduce the noise in the predicted motions, thus preventing from freezing. We further extend the bank to provide explicit priors about the future motions to disambiguate motion prediction, which helps the predictors to generate motions with larger magnitude and higher fidelity than possible before. Extensive experiments on AIST++, a public large-scale 3D dance motion benchmark, demonstrate that our method notably outperforms the baselines in terms of quality, diversity and time length.

PDF Details

JBHI Journal 2021 Journal Article

A Visually Interpretable Deep Learning Framework for Histopathological Image-Based Skin Cancer Diagnosis

Shancheng Jiang
Huichuan Li
Zhi Jin

Owing to the high incidence rate and the severe impact of skin cancer, the precise diagnosis of malignant skin tumors is a significant goal, especially considering treatment is normally effective if the tumor is detected early. Limited published histopathological image sets and the lack of an intuitive correspondence between the features of lesion areas and a certain type of skin cancer pose a challenge to the establishment of high-quality and interpretable computer-aided diagnostic (CAD) systems. To solve this problem, a light-weight attention mechanism-based deep learning framework, namely, DRANet, is proposed to differentiate 11 types of skin diseases based on a real histopathological image set collected by us during the last 10 years. The CAD system can output not only the name of a certain disease but also a visualized diagnostic report showing possible areas related to the disease. The experimental results demonstrate that the DRANet obtains significantly better performance than baseline models (i. e. , InceptionV3, ResNet50, VGG16, and VGG19) with comparable parameter size and competitive accuracy with fewer model parameters. Visualized results produced by the hidden layers of the DRANet actually highlight part of the class-specific regions of diagnostic points and are valuable for decision making in the diagnosis of skin diseases.

Details DOI

AAMAS Conference 2021 Conference Paper

Fast Adaptation to External Agents via Meta Imitation Counterfactual Regret Advantage

Mingyue Zhang
Zhi Jin
Yang Xu
Zehan Shen
Kun Liu
Keyu Pan

This paper focuses on the multi-agent credit assignment problem. We propose a novel multi-agent reinforcement learning algorithm called meta imitation counterfactual regret advantage (MICRA) and a three-phase framework for training, adaptation, and execution of MICRA. The key features are: (1) a counterfactual regret advantage is proposed to optimize the target agents’ policy; (2) a meta-imitator is designed to infer the external agents’ policies. Results show that MICRA outperforms state-of-the-art algorithms.

PDF

NeurIPS Conference 2021 Conference Paper

Integrating Tree Path in Transformer for Code Representation

Han Peng
Ge Li
Wenhan Wang
YunFei Zhao
Zhi Jin

Learning distributed representation of source code requires modelling its syntax and semantics. Recent state-of-the-art models leverage highly structured source code representations, such as the syntax trees and paths therein. In this paper, we investigate two representative path encoding methods shown in previous research work and integrate them into the attention module of Transformer. We draw inspiration from the ideas of positional encoding and modify them to incorporate these path encoding. Specifically, we encode both the pairwise path between tokens of source code and the path from the leaf node to the tree root for each token in the syntax tree. We explore the interaction between these two kinds of paths by integrating them into the unified Transformer framework. The detailed empirical study for path encoding methods also leads to our novel state-of-the-art representation model TPTrans, which finally outperforms strong baselines. Extensive experiments and ablation studies on code summarization across four different languages demonstrate the effectiveness of our approaches. We release our code at \url{https: //github. com/AwdHanPeng/TPTrans}.

PDF Details

AAAI Conference 2020 Conference Paper

Generating Adversarial Examples for Holding Robustness of Source Code Processing Models

Huangzhao Zhang
Zhuo Li
Ge Li
Lei Ma
Yang Liu
Zhi Jin

Automated processing, analysis, and generation of source code are among the key activities in software and system lifecycle. To this end, while deep learning (DL) exhibits a certain level of capability in handling these tasks, the current stateof-the-art DL models still suffer from non-robust issues and can be easily fooled by adversarial attacks. Different from adversarial attacks for image, audio, and natural languages, the structured nature of programming languages brings new challenges. In this paper, we propose a Metropolis-Hastings sampling-based identiﬁer renaming technique, named Metropolis-Hastings Modiﬁer (MHM), which generates adversarial examples for DL models specialized for source code processing. Our in-depth evaluation on a functionality classiﬁcation benchmark demonstrates the effectiveness of MHM in generating adversarial examples of source code. The higher robustness and performance enhanced through our adversarial training with MHM further conﬁrms the usefulness of DL models-based method for future fully automated source code processing.

PDF Details

NeurIPS Conference 2019 Conference Paper

Code Generation as a Dual Task of Code Summarization

Bolin Wei
Ge Li
Xin Xia
Zhiyi Fu
Zhi Jin

Code summarization (CS) and code generation (CG) are two crucial tasks in the field of automatic software development. Various neural network-based approaches are proposed to solve these two tasks separately. However, there exists a specific intuitive correlation between CS and CG, which has not been exploited in previous work. In this paper, we apply the relations between two tasks to improve the performance of both tasks. In other words, exploiting the duality between the two tasks, we propose a dual training framework to train the two tasks simultaneously. In this framework, we consider the dualities on probability and attention weights, and design corresponding regularization terms to constrain the duality. We evaluate our approach on two datasets collected from GitHub, and experimental results show that our dual framework can improve the performance of CS and CG tasks over baselines.

PDF Details

IJCAI Conference 2018 Conference Paper

Summarizing Source Code with Transferred API Knowledge

Xing Hu
Ge Li
Xin Xia
David Lo
Shuai Lu
Zhi Jin

Code summarization, aiming to generate succinct natural language description of source code, is extremely useful for code search and code comprehension. It has played an important role in software maintenance and evolution. Previous approaches generate summaries by retrieving summaries from similar code snippets. However, these approaches heavily rely on whether similar code snippets can be retrieved, how similar the snippets are, and fail to capture the API knowledge in the source code, which carries vital information about the functionality of the source code. In this paper, we propose a novel approach, named TL-CodeSum, which successfully uses API knowledge learned in a different but related task to code summarization. Experiments on large-scale real-world industry Java projects indicate that our approach is effective and outperforms the state-of-the-art in code summarization.

PDF Details

AAAI Conference 2018 Short Paper

Towards Neural Speaker Modeling in Multi-Party Conversation: The Task, Dataset, and Models

Zhao Meng
Lili Mou
Zhi Jin

In this paper, we address the problem of speaker classiﬁcation in multi-party conversation, and collect massive data to facilitate research in this direction. We further investigate temporal-based and content-based models of speakers, and propose several hybrids of them. Experiments show that speaker classiﬁcation is feasible, and that hybrid models outperform each single component. 1

PDF Details

AAAI Conference 2016 Conference Paper

Convolutional Neural Networks over Tree Structures for Programming Language Processing

Lili Mou
Ge Li
Lu Zhang
Tao Wang
Zhi Jin

Programming language processing (similar to natural language processing) is a hot research topic in the ﬁeld of software engineering; it has also aroused growing interest in the artiﬁcial intelligence community. However, different from a natural language sentence, a program contains rich, explicit, and complicated structural information. Hence, traditional NLP models may be inappropriate for programs. In this paper, we propose a novel tree-based convolutional neural network (TBCNN) for programming language processing, in which a convolution kernel is designed over programs’ abstract syntax trees to capture structural information. TBCNN is a generic architecture for programming language processing; our experiments show its effectiveness in two different program analysis tasks: classifying programs according to functionality, and detecting code snippets of certain patterns. TBCNN outperforms baseline methods, including several neural models for NLP.

PDF Details

AAMAS Conference 2010 Conference Paper

Assignment Problem in Requirements Driven Agent Collaboration and its Implementation

Jian Tang
Zhi Jin

Requirements Driven Agent Collaboration (RDAC) is a mechanism where the self-interested service agents actively andautonomously search for the required services submitted bythe request agents and compete to offer the services. Thismechanism would be more suitable for large number of autonomous service providers on internet compared with thecurrent service-oriented computing framework. Collaboration is an important issue in this mechanism. This paper focuses on the collaboration issue in RDAC. First, we define the assignment problem in RDAC and showthat it is NP-complete. Then, we model it as a Kripke structure with normative systems on it. This makes it possibleto bridge the assignment problem and the existing effortsin normative systems, games, mechanisms, etc. Thirdly, anegotiation-based approach is given to solve the problemand the performance of the negotiation is evaluated by simulation. Finally, a tool for RDAC has been implemented toput it into practice and for further testing and evaluation.

PDF

AAMAS Conference 2007 Conference Paper

Requirements Driven Agent Collaboration

Liwei Zheng
Zhi Jin

This paper proposes the requirements driven agent collaboration. This proposal assumes that there are plenty different service agents distributed in Internet. When a request for accomplishing a particular task occurs, these autonomous agents can recognize the newly emergent requirements and dynamically aggregate together to compete with others for fulfilling the requirements. This paper presents a preliminary framework for the requirement driven agent collaboration based on the automated mechanism design.

PDF