Arrow Research search

Author name cluster

Yue Liu

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

51 papers
2 author rows

Possible papers

51

EAAI Journal 2026 Journal Article

Adaptive weighted disentangling variational autoencoder with fine-grained feedback

  • Zhenyao Yu
  • Yue Liu
  • Zitu Liu
  • Zhengwei Yang
  • Yike Guo
  • Qun Liu
  • Guoyin Wang

In the realm of machine learning, the challenge of extracting meaningful low-dimensional structures from high-dimensional data is paramount. Deep learning techniques, particularly Variational Autoencoders (VAE), have proven adept at this task yet often lack semantic interpretability in their representations. To address this issue, disentangled representation learning has been proposed and utilized to learn interpretable representations from data. However, existing methods often rely on heuristic constraints that are manually set and fixed, hindering adaptability and optimization. In this paper, the Adaptive Weighted Disentangling Variational Autoencoder (AwingVAE) is proposed, which introduces a feedback mechanism into the VAE framework, allowing for dynamic parameter optimization and adaptive dimension weighting based on Kullback-Leibler divergence. This feedback mechanism effectively enhances the model's disentanglement, generation, and robustness, with maximum gains of 17. 4%, 7. 565, and 15. 8%, respectively. The proposed method thus offers a new perspective on VAE utilization for representation learning, with extensive evaluations on benchmark datasets supporting its effectiveness. Implementation available at https: //github. com/YuSanTu/AwingVAE.

Details DOI

AAAI Conference 2026 Conference Paper

ExtendAttack: Attacking Servers of LRMs via Extending Reasoning

  • Zhenhao Zhu
  • Yue Liu
  • Zhiwei Xu
  • Yingwei Ma
  • Hongcheng Gao
  • Nuo Chen
  • Yanpei Guo
  • Wenjie Qu

Large Reasoning Models (LRMs) have demonstrated promising performance in complex tasks. However, the resource-consuming reasoning processes may be exploited by attackers to maliciously occupy the resources of the servers, leading to a crash, like the DDoS attack in cyber. To this end, we propose a novel attack method on LRMs termed ExtendAttack to maliciously occupy the resources of servers by stealthily extending the reasoning processes of LRMs. Concretely, we systematically obfuscate characters within a benign prompt, transforming them into a complex, poly-base ASCII representation. This compels the model to perform a series of computationally intensive decoding sub-tasks that are deeply embedded within the semantic structure of the query itself. Extensive experiments demonstrate the effectiveness of our proposed ExtendAttack. Remarkably, it significantly increases response length and latency, with the former increasing by over 2.7 times for the o3 model on the HumanEval benchmark. Besides, it preserves the original meaning of the query and achieves comparable answer accuracy, showing the stealthiness.

PDF Details DOI

AAAI Conference 2026 Conference Paper

Faithful in Steps: Improving Generalization and Citation in RAG via Query Decomposition

  • Yue Liu
  • Zhongying Ru
  • Shimin Di
  • Jipeng Zhang
  • Ruiyuan Zhang
  • Xiaofang Zhou

Retrieval-augment generation is a prevalent strategy to mitigate hallucinations of LLMs. The attributable RAG (RAGQ) generates quotes for its answers. The quotes indicate which input contexts support the RAG to derive the answers, enhancing the answer's verifiability and trustworthiness. However, existing RAGQs exhibit significant degradation when dealing with questions that require multi-hop reasoning and multi-modal understanding, suffering from over-citation, implicit entity identification failure, and poor generalization. In this paper, we propose a novel RAGQ framework, namely QDRAG. QDRAG breaks down the input question into atomic subquestions to identify the implicit entities. Then, the reranker prunes context distractors to eliminate the downstream over-citation. To facilitate query decomposition, we propose two zero-shot approaches: QD-C and QD-R, which guide the QD MLLM to decompose the question based on context knowledge and retrieval rewards, respectively. One interesting finding is that finetuning on the QD task shows better generalizability compared to directly finetuning on the downstream RAGQ task. Experiments on four multi-modal QA benchmarks demonstrate QDRAG's efficacy in grounding answers and generating faithful citations. The framework significantly outperforms all the baselines on both in-domain and out-of-domain tests, even surpassing Gemini-Pro.

PDF Details DOI

AAAI Conference 2026 Conference Paper

RAC-DMVC: Reliability-Aware Contrastive Deep Multi-View Clustering Under Multi-Source Noise

  • Shihao Dong
  • Yue Liu
  • Xiaotong Zhou
  • Yuhui Zheng
  • Huiying Xu
  • Xinzhong Zhu

Multi-view clustering (MVC), which aims to separate the multi-view data into distinct clusters in an unsupervised manner, is a fundamental yet challenging task. To enhance its applicability in real-world scenarios, this paper addresses a more challenging task: MVC under multi-source noises, including missing noise and observation noise. To this end, we propose a novel framework, Reliability-Aware Contrastive Deep Multi-View Clustering (RAC-DMVC), which constructs a reliability graph to guide robust representation learning under noisy environments. Specifically, to address observation noise, we introduce a cross-view reconstruction to enhances robustness at the data level, and a reliability-aware noise contrastive learning to mitigates bias in positive and negative pairs selection caused by noisy representations. To handle missing noise, we design a dual-attention imputation to capture shared information across views while preserving view-specific features. In addition, a self-supervised cluster distillation module further refines the learned representations and improves the clustering performance. Extensive experiments on five benchmark datasets demonstrate that RAC-DMVC outperforms SOTA methods on multiple evaluation metrics and maintains excellent performance under varying ratios of noise.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

Afterburner: Reinforcement Learning Facilitates Self-Improving Code Efficiency Optimization

  • Mingzhe Du
  • Anh Tuan Luu
  • Yue Liu
  • Yuhao Qing
  • Dong Huang
  • Xinyi He
  • Qian Liu
  • Zejun Ma

Large Language Models (LLMs) generate functionally correct solutions but often fall short in code efficiency, a critical bottleneck for real-world deployment. In this paper, we introduce a novel test-time iterative optimization framework to address this, employing a closed-loop system where LLMs iteratively refine code based on empirical performance feedback from an execution sandbox. We explore three training strategies: Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Group Relative Policy Optimization~(GRPO). Experiments on our Venus dataset and the APPS benchmark show that SFT and DPO rapidly saturate in efficiency gains. In contrast, GRPO, using reinforcement learning (RL) with execution feedback, continuously optimizes code performance, significantly boosting both pass@1 (from 47% to 62%) and the likelihood of outperforming human submissions in efficiency (from 31% to 45%). Our work demonstrates effective test-time code efficiency improvement and critically reveals the power of RL in teaching LLMs to truly self-improve code efficiency. We released our code and data at https: //github. com/Elfsong/Afterburner.

PDF Details

AAAI Conference 2025 Conference Paper

Alleviating Performance Disparity in Adversarial Spatiotemporal Graph Learning Under Zero-Inflated Distribution

  • Songran Bai
  • Yuheng Ji
  • Yue Liu
  • Xingwei Zhang
  • Xiaolong Zheng
  • Daniel Dajun Zeng

Spatiotemporal Graph Learning (SGL) under Zero-Inflated Distribution (ZID) is crucial for urban risk management tasks, including crime prediction and traffic accident profiling. However, SGL models are vulnerable to adversarial attacks, compromising their practical utility. While adversarial training (AT) has been widely used to bolster model robustness, our study finds that traditional AT exacerbates performance disparities between majority and minority classes under ZID, potentially leading to irreparable losses due to underreporting critical risk events. In this paper, we first demonstrate the smaller top-k gradients and lower separability of minority class are key factors contributing to this disparity. To address these issues, we propose MinGRE, a framework for Minority Class Gradients and Representations Enhancement. MinGRE employs a multi-dimensional attention mechanism to reweight spatiotemporal gradients, minimizing the gradient distribution discrepancies across classes. Additionally, we introduce an uncertainty-guided contrastive loss to improve the inter-class separability and intra-class compactness of minority representations with higher uncertainty. Extensive experiments demonstrate that the MinGRE framework not only significantly reduces the performance disparity across classes but also achieves enhanced robustness compared to existing baselines. These findings underscore the potential of our method in fostering the development of more equitable and robust models.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

DGCBench: A Deep Graph Clustering Benchmark

  • Benyu Wu
  • Yue Liu
  • Qiaoyu Tan
  • Xinwang Liu
  • Wei Du
  • Jun Wang
  • Guoxian Yu

Deep graph clustering (DGC) aims to partition graph nodes into distinct clusters in an unsupervised manner. Despite rapid advancements in this field, DGC remains inherently challenging due to the absence of ground-truth, which complicates the design of effective algorithms and impedes the establishment of standardized benchmarks. The lack of unified datasets, evaluation protocols, and metrics further exacerbates these challenges, making it difficult to systematically assess and compare DGC methods. To address these limitations, we introduce $\texttt{DGCBench}$, the first comprehensive and unified benchmark for DGC methods. It evaluates 12 state-of-the-art DGC methods across 12 datasets from diverse domains and scales, spanning 6 critical dimensions: $\textbf{discriminability}$, $\textbf{effectiveness}$, $\textbf{scalability}$, $\textbf{efficiency}$, $\textbf{stability}$, and $\textbf{robustness}$. Additionally, we develop $\texttt{PyDGC}$, an open-source Python library that standardizes the DGC training and evaluation paradigm. Through systematic experiments, we reveal persistent limitations in existing methods, specifically regarding the homophily bottleneck, training instability, vulnerability to perturbations, efficiency plateau, scalability challenges, and poor discriminability, thereby offering actionable insights for future research. We hope that $\texttt{DGCBench}$, $\texttt{PyDGC}$, and our analyses will collectively accelerate the progress in the DGC community. The code is available at https: //github. com/Marigoldwu/PyDGC.

PDF Details

ICML Conference 2025 Conference Paper

Fairness on Principal Stratum: A New Perspective on Counterfactual Fairness

  • Haoxuan Li 0001
  • Zeyu Tang 0002
  • Zhichao Jiang
  • Zhuangyan Fang
  • Yue Liu
  • Zhi Geng
  • Kun Zhang 0001

Fairness in human and algorithmic decision-making is crucial in areas such as criminal justice, education, and social welfare. Recently, counterfactual fairness has drawn increasing research interest, suggesting that decision-making for individuals should remain the same when intervening with different values on protected attributes. Nevertheless, the question of "which attributes and individuals should be protected" is rarely discussed in the existing counterfactual fairness literature. For example, when considering leg disability as a protected attribute, the algorithms should not treat individuals with leg disabilities differently in college admissions, but one may naturally consider this factor when selecting runner athletes. In other words, when and how to enforce fairness is expected to depend on the causal relation between the protected attribute and the outcome of interest. Formally, this paper proposes principal counterfactual fairness using the concept of principal stratification from the causal inference literature, focusing on whether an algorithm is counterfactually fair for individuals whose protected attribute has no individual causal effect on the outcome of interest. To examine whether an algorithm satisfies principal counterfactual fairness, we derive the statistical bounds and propose a post-processing approach to achieving principal counterfactual fairness with minimal individual decision changes. Experiments are conducted using synthetic and real-world datasets to verify the effectiveness of our methods.

Details

NeurIPS Conference 2025 Conference Paper

GuardReasoner-VL: Safeguarding VLMs via Reinforced Reasoning

  • Yue Liu
  • Shengfang Zhai
  • Mingzhe Du
  • Yulin Chen
  • Tri Cao
  • Hongcheng Gao
  • Cheng Wang
  • Xinfeng Li

To enhance the safety of VLMs, this paper introduces a novel reasoning-based VLM guard model dubbed GuardReasoner-VL. The core idea is to incentivize the guard model to deliberatively reason before making moderation decisions via online RL. First, we construct GuardReasoner-VLTrain, a reasoning corpus with 123K samples and 631K reasoning steps, spanning text, image, and text-image inputs. Then, based on it, we cold-start our model's reasoning ability via SFT. In addition, we further enhance reasoning regarding moderation through online RL. Concretely, to enhance diversity and difficulty of samples, we conduct rejection sampling followed by data augmentation via the proposed safety-aware data concatenation. Besides, we use a dynamic clipping parameter to encourage exploration in early stages and exploitation in later stages. To balance performance and token efficiency, we design a length-aware safety reward that integrates accuracy, format, and token cost. Extensive experiments demonstrate the superiority of our model. Remarkably, it surpasses the runner-up by 19. 27% F1 score on average, as shown in Figure 1. We release data, code, and models (3B/7B) of GuardReasoner-VL: https: //github. com/yueliu1999/GuardReasoner-VL.

PDF Details

NeurIPS Conference 2025 Conference Paper

HYPERION: Fine-Grained Hypersphere Alignment for Robust Federated Graph Learning

  • Frank Wan
  • Xiaoran Shang
  • Yuxin Wu
  • Guibin Zhang
  • Jinhe Bi
  • Liangtao Zheng
  • Xin Lin
  • Yue Liu

Robust Federated Graph Learning (FGL) provides an effective decentralized framework for training Graph Neural Networks (GNNs) in noisy-label environments. However, the subtlety of noise during training presents formidable obstacles for developing robust FGL systems. Previous robust FL approaches neither adequately constrain edge-mediated error propagation nor account for intra-class topological differences. At the client level, we innovatively demonstrate that hyperspherical embedding can effectively capture graph structures in a fine-grained manner. Correspondingly, our method effectively addresses the aforementioned issues through fine-grained hypersphere alignment. Moreover, we uncover undetected noise arising from localized perspective constraints and propose the geometric-aware hyperspherical purification module at the server level. Combining both level strategies, we present our robust FGL framework, **HYPERION**, which operates all components within a unified hyperspherical space. **HYPERION** demonstrates remarkable robustness across multiple datasets, for instance, achieving a 29. 7\% $\uparrow$ F1-macro score with 50\%-pair noise on Cora. The code is available for anonymous access at \url{https: //anonymous. 4open. science/r/Hyperion-NeurIPS/}.

PDF Details

EAAI Journal 2025 Journal Article

Knowledge Augmented Expert finding framework via knowledge graph embedding for Community Question Answering

  • Yue Liu
  • Zitu Liu
  • Zhenyao Yu
  • Qingshan Fu
  • Weize Tang
  • Wenxuan Yao
  • Zhibin Sun

Expert Finding in Community Question Answering aims to recommend appropriate experts to answer posted questions. However, existing approaches focus on exploiting semantic extraction and authority analysis techniques, failing to realize the latent knowledge-aspect connections between questions and experts. Therefore, the experts recommended for posted questions are limited to text-to-text approximation or domain-independent authority. In this study, we propose a Knowledge Augmented Expert finding framework (KAExpert) that introduces knowledge level information into semantic-based and authority-based expert finding method. A community knowledge graph is firstly constructed by acquiring ternary relations from public databases based on question tags and knowledge-related phrases. Then a flexible knowledge graph embedding is designed to extract the matching relationship between questions and experts at the knowledge level. Along this line, the knowledge-level authority is calculated based on the knowledge graph embedding to optimize the results of domain matching. According to knowledge graph embedding and knowledge-level authority, KAExpert is constructed to optimize the results of domain matching so that the found experts set consists of high-level expert with semantic matching and knowledge matching. Finally, experimental results on two real-world datasets collected from two major commercial question answering web sites show that KAExpert can outperform baseline methods with a significant margin.

Details DOI

NeurIPS Conference 2025 Conference Paper

MLR-Bench: Evaluating AI Agents on Open-Ended Machine Learning Research

  • Hui Chen
  • Miao Xiong
  • Yujie Lu
  • Wei Han
  • Ailin Deng
  • Yufei He
  • Jiaying Wu
  • Yibo Li

Recent advancements in AI agents have demonstrated their growing potential to drive and support scientific discovery. In this work, we introduce MLR-Bench, a comprehensive benchmark for evaluating AI agents on open-ended machine learning research. MLR-Bench includes three key components: (1) 201 research tasks sourced from NeurIPS, ICLR, and ICML workshops covering diverse ML topics; (2) MLR-Judge, an automated evaluation framework combining LLM-based reviewers with carefully designed review rubrics to assess research quality; and (3) MLR-Agent, a modular agent scaffold capable of completing research tasks through four stages: idea generation, proposal formulation, experimentation, and paper writing. Our framework supports both stepwise assessment across these distinct research stages, and end-to-end evaluation of the final research paper. We then use MLR-Bench to evaluate six frontier LLMs and an advanced coding agent, finding that while LLMs are effective at generating coherent ideas and well-structured papers, current coding agents frequently (e. g. , in 80\% of the cases) produce fabricated or invalidated experimental results—posing a major barrier to scientific reliability. We validate MLR-Judge through human evaluation, showing high agreement with expert reviewers, supporting its potential as a scalable tool for research evaluation. We open-source MLR-Bench to help the community benchmark, diagnose, and improve AI research agents toward trustworthy and transparent scientific discovery.

PDF Details

EAAI Journal 2025 Journal Article

Multiple scales fusion and query matching stabilization for detection with transformer

  • Shenyu Du
  • Xijun Liang
  • Kun Wu
  • Ye Tian
  • Yue Liu
  • Ling Jian

Recent advances in object detection with Transformer-based models like Detection with Transformer (DETR) have improved performance, but challenges remain with multi-scale fused features. These features introduce redundant tokens and bias toward larger objects, slowing down training. To overcome these issues, we propose two novel encoders: the Similarity-based Deduplication Encoder (SDE) and the Hybrid Multi-object Encoder (HMoE). HMoE employs an offset-based attention window to enhance local attention for objects of varying sizes across feature maps, while SDE reduces redundancy by calculating attention scores across multiple scales. Additionally, we introduce a One-to-many Positive Matching (OmPM) strategy to improve query stability. OmPM generates query vectors from multiple positive samples, resulting in more diverse and semantically meaningful queries. Our model demonstrates substantial performance improvements. On the Visual Object Classes Challenge 2007 dataset, it achieves a +5. 04 mean Average Precision (mAP) and +5. 1 Average Precision for small objects (APs) for small objects in just 24 epochs. On the Microsoft Common Objects in Context (COCO) dataset, the model reaches 50. 1 mAP and 34. 2 APs in only 8 epochs, and 52. 4 mAP and 35. 6 APs in 24 epochs. This significantly accelerates convergence, reducing training time by 66% compared to benchmarks while maintaining or exceeding detection accuracy. Furthermore, our model achieves 27 Frames Per Second (FPS) on the COCO dataset, setting a new record among DETR-like methods with high detection accuracies.

Details DOI

JMLR Journal 2025 Journal Article

On the Representation of Pairwise Causal Background Knowledge and Its Applications in Causal Inference

  • Zhuangyan Fang
  • Ruiqi Zhao
  • Yue Liu
  • Yangbo He

Pairwise causal background knowledge about the existence or absence of causal edges and paths is frequently encountered in observational studies. Such constraints allow the shared directed and undirected edges in the constrained subclass of Markov equivalent DAGs to be represented as a causal maximally partially directed acyclic graph (MPDAG). In this paper, we first provide a sound and complete graphical characterization of causal MPDAGs and introduce a minimal representation of a causal MPDAG. Then, we give a unified representation for three types of pairwise causal background knowledge, including direct, ancestral and non-ancestral causal knowledge, by introducing a novel concept called direct causal clause (DCC). Using DCCs, we study the consistency and equivalence of pairwise causal background knowledge and show that any pairwise causal background knowledge set can be uniquely and equivalently decomposed into the causal MPDAG representing the refined Markov equivalence class and a minimal residual set of DCCs. Polynomial-time algorithms are also provided for checking consistency and equivalence, as well as for finding the decomposed MPDAG and the residual DCCs. Finally, with pairwise causal background knowledge, we prove a sufficient and necessary condition to identify causal effects and surprisingly find that the identifiability of causal effects only depends on the decomposed MPDAG. We also develop a local IDA-type algorithm to estimate the possible values of an unidentifiable effect. Simulations suggest that pairwise causal background knowledge can significantly improve the identifiability of causal effects. [abs] [ pdf ][ bib ] &copy JMLR 2025. ( edit, beta )

PDF Details

JBHI Journal 2025 Journal Article

Super-Resolution Reconstruction of OCTA Via Multi-Field-of-View Representation Learning

  • Huaying Hao
  • Shaoyi Leng
  • Yanda Meng
  • Yonghuai Liu
  • Yalin Zheng
  • Huazhu Fu
  • Jiong Zhang
  • Quanyong Yi

High-resolution Optical Coherence Tomography Angiography (OCTA) images are essential for morphological analysis and biomarker measurement of the retinal vasculature. They can also provide underlying biomarkers for the accurate analysis of eye-related diseases. The trade-off between the high resolution (HR) and large scanning field-of-view (FOV) is a long-standing problem for OCTA image instrument. A large FOV image provides more retinal information with shorter acquisition time but often suffers from low resolution (LR), high scatter noise, and poor vascular contrast. In order to obtain HR OCTA images with larger FOV, we propose a novel self-similar dynamic domain adaptation network based on cross-field-of-view representation learning. The network enables LR images ( i. e. , $6\times \text{6}~\text{mm}^{2}$ ) to learn HR image ( i. e. , $3\times \text{3}~\text{mm}^{2}$ ) feature representations specialized for OCTA by constructing feature mapping relations for cross-field-of-view OCTA scans. To be specific, a multiple random degradation model is proposed on HR images to generate various synthetic LR images. Further, we propose a dynamic domain adaptation framework that prompts feature dynamic alignment of the LR image reconstruction results with those of synthetic LR images. Finally, a novel self-similar supervision loss is proposed to optimize the reconstruction results from LR to HR by exploiting the similarity between vessels in different regions. Experimental results on three OCTA datasets show that the proposed method surpasses existing state-of-the-art ones, significantly enhancing retinal structure segmentation and disease classification. Our OCTA dataset (the first dataset in this research area with paired $3\times 3$ and $6\times \text{6}~\text{mm}^{2}$ OCTA images) and code are publicly available.

Details DOI

NeurIPS Conference 2025 Conference Paper

VisualLens: Personalization through Task-Agnostic Visual History

  • Wang Bill Zhu
  • Deqing Fu
  • Kai Sun
  • Yi Lu
  • Zhaojiang Lin
  • Seungwhan Moon
  • Kanika Narang
  • Mustafa Canim

Existing recommendation systems either rely on user interaction logs, such as online shopping history for shopping recommendations, or focus on text signals. However, item-based histories are not always accessible and generalizable for multimodal recommendation. We hypothesize that a user's visual history --- comprising images from daily life --- can offer rich, task-agnostic insights into their interests and preferences, and thus be leveraged for effective personalization. To this end, we propose VisualLens, a novel framework that leverages multimodal large language models (MLLMs) to enable personalization using task-agnostic visual history. VisualLens extracts, filters, and refines a spectrum user profile from the visual history to support personalized recommendation. We created two new benchmarks, Google-Review-V and Yelp-V, with task-agnostic visual histories, and show that VisualLens improves over state-of-the-art item-based multimodal recommendations by 5-10\% on Hit@3, and outperforms GPT-4o by 2-5\%. Further analysis shows that VisualLens is robust across varying history lengths and excels at adapting to both longer histories and unseen content categories.

PDF Details

NeurIPS Conference 2024 Conference Paper

A Local Method for Satisfying Interventional Fairness with Partially Known Causal Graphs

  • Haoxuan Li
  • Yue Liu
  • Zhi Geng
  • Kun Zhang

Developing fair automated machine learning algorithms is critical in making safe and trustworthy decisions. Many causality-based fairness notions have been proposed to address the above issues by quantifying the causal connections between sensitive attributes and decisions, and when the true causal graph is fully known, certain algorithms that achieve interventional fairness have been proposed. However, when the true causal graph is unknown, it is still challenging to effectively and efficiently exploit partially directed acyclic graphs (PDAGs) to achieve interventional fairness. To exploit the PDAGs for achieving interventional fairness, previous methods have been built on variable selection or causal effect identification, but limited to reduced prediction accuracy or strong assumptions. In this paper, we propose a general min-max optimization framework that can achieve interventional fairness with promising prediction accuracy and can be extended to maximally oriented PDAGs (MPDAGs) with added background knowledge. Specifically, we first estimate all possible treatment effects of sensitive attributes on a given prediction model from all possible adjustment sets of sensitive attributes via an efficient local approach. Next, we propose to alternatively update the prediction model and possible estimated causal effects, where the prediction model is trained via a min-max loss to control the worst-case fairness violations. Extensive experiments on synthetic and real-world datasets verify the superiority of our methods. To benefit the research community, we have released our project at https: //github. com/haoxuanli-pku/NeurIPS24-Interventional-Fairness-with-PDAGs.

PDF Details DOI

ICLR Conference 2024 Conference Paper

At Which Training Stage Does Code Data Help LLMs Reasoning?

  • Yingwei Ma
  • Yue Liu
  • Yue Yu 0001
  • Yuanliang Zhang
  • Yu Jiang 0001
  • Changjian Wang
  • Shanshan Li 0001

Large Language models (LLMs) have exhibited remarkable reasoning capabilities and become the foundation of language technologies. Inspired by the great success of code data in training LLMs, we naturally wonder at which training stage introducing code data can really help LLMs reasoning. To this end, this paper systematically explores the impact of code data on LLMs at different stages. Concretely, we introduce the code data at the pre-training stage, instruction-tuning stage, and both of them, respectively. Then, the reasoning capability of LLMs is comprehensively and fairly evaluated via six reasoning tasks. We critically analyze the experimental results and provide conclusions with insights. First, pre-training LLMs with the mixture of code and text can significantly enhance LLMs' general reasoning capability almost without negative transfer on other tasks. Besides, at the instruction-tuning stage, code data endows LLMs the task-specific reasoning capability. Moreover, the dynamic mixing strategy of code and text data assists LLMs to learn reasoning capability step-by-step during training. These insights deepen the understanding of LLMs regarding reasoning ability for their application, such as scientific question answering, legal support, etc.

Details

NeurIPS Conference 2024 Conference Paper

Clustering then Propagation: Select Better Anchors for Knowledge Graph Embedding

  • Ke Liang
  • Yue Liu
  • Hao Li
  • Lingyuan Meng
  • Suyuan Liu
  • Siwei Wang
  • Sihang Zhou
  • Xinwang Liu

Traditional knowledge graph embedding (KGE) models map entities and relations to unique embedding vectors in a shallow lookup manner. As the scale of data becomes larger, this manner will raise unaffordable computational costs. Anchor-based strategies have been treated as effective ways to alleviate such efficiency problems by propagation on representative entities instead of the whole graph. However, most existing anchor-based KGE models select the anchors in a primitive manner, which limits their performance. To this end, we propose a novel anchor-based strategy for KGE, i. e. , a relational clustering-based anchor selection strategy (RecPiece), where two characteristics are leveraged, i. e. , (1) representative ability of the cluster centroids and (2) descriptive ability of relation types in KGs. Specifically, we first perform clustering over features of factual triplets instead of entities, where cluster number is naturally set as number of relation types since each fact can be characterized by its relation in KGs. Then, representative triplets are selected around the clustering centroids, further mapped into corresponding anchor entities. Extensive experiments on six datasets show that RecPiece achieves higher performances but comparable or even fewer parameters compared to previous anchor-based KGE models, indicating that our model can select better anchors in a more scalable way.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

CRAG - Comprehensive RAG Benchmark

  • Xiao Yang
  • Kai Sun
  • Hao Xin
  • Yushi Sun
  • Nikita Bhalla
  • Xiangsen Chen
  • Sajal Choudhary
  • Rongze D. Gui

Retrieval-Augmented Generation (RAG) has recently emerged as a promising solution to alleviate Large Language Model (LLM)’s deficiency in lack of knowledge. Existing RAG datasets, however, do not adequately represent the diverse and dynamic nature of real-world Question Answering (QA) tasks. To bridge this gap, we introduce the Comprehensive RAG Benchmark (CRAG), a factual question answering benchmark of 4, 409 question-answer pairs and mock APIs to simulate web and Knowledge Graph (KG) search. CRAG is designed to encapsulate a diverse array of questions across five domains and eight question categories, reflecting varied entity popularity from popular to long-tail, and temporal dynamisms ranging from years to seconds. Our evaluation on this benchmark highlights the gap to fully trustworthy QA. Whereas most advanced LLMs achieve $\le 34\%$ accuracy on CRAG, adding RAG in a straightforward manner improves the accuracy only to 44%. State-of-the-art industry RAG solutions only answer 63% questions without any hallucination. CRAG also reveals much lower accuracy in answering questions regarding facts with higher dynamism, lower popularity, or higher complexity, suggesting future research directions. The CRAG benchmark laid the groundwork for a KDD Cup 2024 challenge, attracted thousands of participants and submissions. We commit to maintaining CRAG to serve research communities in advancing RAG solutions and general QA solutions. CRAG is available at https: //github. com/facebookresearch/CRAG/.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Cross-Gate MLP with Protein Complex Invariant Embedding Is a One-Shot Antibody Designer

  • Cheng Tan
  • Zhangyang Gao
  • Lirong Wu
  • Jun Xia
  • Jiangbin Zheng
  • Xihong Yang
  • Yue Liu
  • Bozhen Hu

Antibodies are crucial proteins produced by the immune system in response to foreign substances or antigens. The specificity of an antibody is determined by its complementarity-determining regions (CDRs), which are located in the variable domains of the antibody chains and form the antigen-binding site. Previous studies have utilized complex techniques to generate CDRs, but they suffer from inadequate geometric modeling. Moreover, the common iterative refinement strategies lead to an inefficient inference. In this paper, we propose a simple yet effective model that can co-design 1D sequences and 3D structures of CDRs in a one-shot manner. To achieve this, we decouple the antibody CDR design problem into two stages: (i) geometric modeling of protein complex structures and (ii) sequence-structure co-learning. We develop a novel macromolecular structure invariant embedding, typically for protein complexes, that captures both intra- and inter-component interactions among the backbone atoms, including Calpha, N, C, and O atoms, to achieve comprehensive geometric modeling. Then, we introduce a simple cross-gate MLP for sequence-structure co-learning, allowing sequence and structure representations to implicitly refine each other. This enables our model to design desired sequences and structures in a one-shot manner. Extensive experiments are conducted to evaluate our results at both the sequence and structure level, which demonstrate that our model achieves superior performance compared to the state-of-the-art antibody CDR design methods.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

End-to-end Learnable Clustering for Intent Learning in Recommendation

  • Yue Liu
  • Shihao Zhu
  • Jun Xia
  • Yingwei Ma
  • Jian Ma
  • Xinwang Liu
  • Shengju Yu
  • Kejun Zhang

Intent learning, which aims to learn users' intents for user understanding and item recommendation, has become a hot research spot in recent years. However, existing methods suffer from complex and cumbersome alternating optimization, limiting performance and scalability. To this end, we propose a novel intent learning method termed \underline{ELCRec}, by unifying behavior representation learning into an \underline{E}nd-to-end \underline{L}earnable \underline{C}lustering framework, for effective and efficient \underline{Rec}ommendation. Concretely, we encode user behavior sequences and initialize the cluster centers (latent intents) as learnable neurons. Then, we design a novel learnable clustering module to separate different cluster centers, thus decoupling users' complex intents. Meanwhile, it guides the network to learn intents from behaviors by forcing behavior embeddings close to cluster centers. This allows simultaneous optimization of recommendation and clustering via mini-batch data. Moreover, we propose intent-assisted contrastive learning by using cluster centers as self-supervision signals, further enhancing mutual promotion. Both experimental results and theoretical analyses demonstrate the superiority of ELCRec from six perspectives. Compared to the runner-up, ELCRec improves NDCG@5 by 8. 9\% and reduces computational costs by 22. 5\% on the Beauty dataset. Furthermore, due to the scalability and universal applicability, we deploy this method on the industrial recommendation system with 130 million page views and achieve promising results. The codes are available on GitHub\footnote{https: //github. com/yueliu1999/ELCRec}. A collection (papers, codes, datasets) of deep group recommendation/intent learning methods is available on GitHub\footnote{https: //github. com/yueliu1999/Awesome-Deep-Group-Recommendation}.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

FlexMol: A Flexible Toolkit for Benchmarking Molecular Relational Learning

  • Sizhe Liu
  • Jun Xia
  • Lecheng Zhang
  • Yuchen Liu
  • Yue Liu
  • Wenjie Du
  • Zhangyang Gao
  • Bozhen Hu

Molecular relational learning (MRL) is crucial for understanding the interaction behaviors between molecular pairs, a critical aspect of drug discovery and development. However, the large feasible model space of MRL poses significant challenges to benchmarking, and existing MRL frameworks face limitations in flexibility and scope. To address these challenges, avoid repetitive coding efforts, and ensure fair comparison of models, we introduce FlexMol, a comprehensive toolkit designed to facilitate the construction and evaluation of diverse model architectures across various datasets and performance metrics. FlexMol offers a robust suite of preset model components, including 16 drug encoders, 13 protein sequence encoders, 9 protein structure encoders, and 7 interaction layers. With its easy-to-use API and flexibility, FlexMol supports the dynamic construction of over 70, 000 distinct combinations of model architectures. Additionally, we provide detailed benchmark results and code examples to demonstrate FlexMol’s effectiveness in simplifying and standardizing MRL model development and comparison. FlexMol is open-sourced and available at https: //github. com/Steven51516/FlexMol.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Hawkes-Enhanced Spatial-Temporal Hypergraph Contrastive Learning Based on Criminal Correlations

  • Ke Liang
  • Sihang Zhou
  • Meng Liu
  • Yue Liu
  • Wenxuan Tu
  • Yi Zhang
  • Liming Fang
  • Zhe Liu

Crime prediction is a crucial yet challenging task within urban computing, which benefits public safety and resource optimization. Over the years, various models have been proposed, and spatial-temporal hypergraph learning models have recently shown outstanding performances. However, three correlations underlying crime are ignored, thus hindering the performance of previous models. Specifically, there are two spatial correlations and one temporal correlation, i.e., (1) co-occurrence of different types of crimes (type spatial correlation), (2) the closer to the crime center, the more dangerous it is around the neighborhood area (neighbor spatial correlation), and (3) the closer between two timestamps, the more relevant events are (hawkes temporal correlation). To this end, we propose Hawkes-enhanced Spatial-Temporal Hypergraph Contrastive Learning framework (HCL), which mines the aforementioned correlations via two specific strategies. Concretely, contrastive learning strategies are designed for two spatial correlations, and hawkes process modeling is adopted for temporal correlations. Extensive experiments demonstrate the promising capacities of HCL from four aspects, i.e., superiority, transferability, effectiveness, and sensitivity.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Identify Then Recommend: Towards Unsupervised Group Recommendation

  • Yue Liu
  • Shihao Zhu
  • Tianyuan Yang
  • Jian Ma
  • Wenliang Zhong

Group Recommendation (GR), which aims to recommend items to groups of users, has become a promising and practical direction for recommendation systems. This paper points out two issues of the state-of-the-art GR models. (1) The pre-defined and fixed number of user groups is inadequate for real-time industrial recommendation systems, where the group distribution can shift dynamically. (2) The training schema of existing GR methods is supervised, necessitating expensive user-group and group-item labels, leading to significant annotation costs. To this end, we present a novel unsupervised group recommendation framework named $\underline{\text{I}}$dentify $\underline{\text{T}}$hen $\underline{\text{R}}$ecommend ($\underline{\text{ITR}}$), where it first identifies the user groups in an unsupervised manner even without the pre-defined number of groups, and then two pre-text tasks are designed to conduct self-supervised group recommendation. Concretely, at the group identification stage, we first estimate the adaptive density of each user point, where areas with higher densities are more likely to be recognized as group centers. Then, a heuristic merge-and-split strategy is designed to discover the user groups and decision boundaries. Subsequently, at the self-supervised learning stage, the pull-and-repulsion pre-text task is proposed to optimize the user-group distribution. Besides, the pseudo group recommendation pre-text task is designed to assist the recommendations. Extensive experiments demonstrate the superiority and effectiveness of ITR on both user recommendation (e. g. , 22. 22\% NDCG@5 $\uparrow$) and group recommendation (e. g. , 22. 95\% NDCG@5 $\uparrow$). Furthermore, we deploy ITR on the industrial recommender and achieve promising results.

PDF Details DOI

TMLR Journal 2024 Journal Article

In-context Learning with Retrieved Demonstrations for Language Models: A Survey

  • Man Luo
  • Xin Xu
  • Yue Liu
  • Panupong Pasupat
  • Mehran Kazemi

Large language models have demonstrated remarkable few-shot in-context learning (ICL) capabilities, adapting to new tasks with few-shots demonstrations. However, the efficacy of ICL is highly dependent on the selection of these demonstrations. Recent developments have introduced retrieval-based in-context learning (RetICL), which dynamically retrieves demonstrations tailored to each input query. This approach leverages existing databases and retrieval systems, enhancing efficiency and scalability while mitigating biases inherent in manual example selection. Given the promising results and growing interest in RetICL, we present a comprehensive survey of this field. Our review encompasses: design choices for ICL demonstration retrieval models, retrieval training procedures, inference strategies and current applications of RetICL. In the end, we explore future directions for this emerging technology.

PDF Details

NeurIPS Conference 2024 Conference Paper

Learning Complete Protein Representation by Dynamically Coupling of Sequence and Structure

  • Bozhen Hu
  • Cheng Tan
  • Jun Xia
  • Yue Liu
  • Lirong Wu
  • Jiangbin Zheng
  • Yongjie Xu
  • Yufei Huang

Learning effective representations is imperative for comprehending proteins and deciphering their biological functions. Recent strides in language models and graph neural networks have empowered protein models to harness primary or tertiary structure information for representation learning. Nevertheless, the absence of practical methodologies to appropriately model intricate inter-dependencies between protein sequences and structures has resulted in embeddings that exhibit low performance on tasks such as protein function prediction. In this study, we introduce CoupleNet, a novel framework designed to interlink protein sequences and structures to derive informative protein representations. CoupleNet integrates multiple levels and scales of features in proteins, encompassing residue identities and positions for sequences, as well as geometric representations for tertiary structures from both local and global perspectives. A two-type dynamic graph is constructed to capture adjacent and distant sequential features and structural geometries, achieving completeness at the amino acid and backbone levels. Additionally, convolutions are executed on nodes and edges simultaneously to generate comprehensive protein embeddings. Experimental results on benchmark datasets showcase that CoupleNet outperforms state-of-the-art methods, exhibiting particularly superior performance in low-sequence similarities scenarios, adeptly identifying infrequently encountered functions and effectively capturing remote homology relationships in proteins.

PDF Details DOI

AAAI Conference 2024 Conference Paper

MINES: Message Intercommunication for Inductive Relation Reasoning over Neighbor-Enhanced Subgraphs

  • Ke Liang
  • Lingyuan Meng
  • Sihang Zhou
  • Wenxuan Tu
  • Siwei Wang
  • Yue Liu
  • Meng Liu
  • Long Zhao

GraIL and its variants have shown their promising capacities for inductive relation reasoning on knowledge graphs. However, the uni-directional message-passing mechanism hinders such models from exploiting hidden mutual relations between entities in directed graphs. Besides, the enclosing subgraph extraction in most GraIL-based models restricts the model from extracting enough discriminative information for reasoning. Consequently, the expressive ability of these models is limited. To address the problems, we propose a novel GraIL-based framework, termed MINES, by introducing a Message Intercommunication mechanism on the Neighbor-Enhanced Subgraph. Concretely, the message intercommunication mechanism is designed to capture the omitted hidden mutual information. It introduces bi-directed information interactions between connected entities by inserting an undirected/bi-directed GCN layer between uni-directed RGCN layers. Moreover, inspired by the success of involving more neighbors in other graph-based tasks, we extend the neighborhood area beyond the enclosing subgraph to enhance the information collection for inductive relation reasoning. Extensive experiments prove the promising capacity of the proposed MINES from various aspects, especially for the superiority, effectiveness, and transfer ability.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

NovoBench: Benchmarking Deep Learning-based \emph{De Novo} Sequencing Methods in Proteomics

  • Jingbo Zhou
  • Shaorong Chen
  • Jun Xia
  • Sizhe Sizhe Liu
  • Tianze Ling
  • Wenjie Du
  • Yue Liu
  • Jianwei Yin

Tandem mass spectrometry has played a pivotal role in advancing proteomics, enabling the analysis of protein composition in biological tissues. Many deep learning methods have been developed for \emph{de novo} peptide sequencing task, i. e. , predicting the peptide sequence for the observed mass spectrum. However, two key challenges seriously hinder the further research of this important task. Firstly, since there is no consensus for the evaluation datasets, the empirical results in different research papers are often not comparable, leading to unfair comparison. Secondly, the current methods are usually limited to amino acid-level or peptide-level precision and recall metrics. In this work, we present the first unified benchmark NovoBench for \emph{de novo} peptide sequencing, which comprises diverse mass spectrum data, integrated models, and comprehensive evaluation metrics. Recent impressive methods, including DeepNovo, PointNovo, Casanovo, InstaNovo, AdaNovo and $\pi$-HelixNovo are integrated into our framework. In addition to amino acid-level and peptide-level precision and recall, we also evaluate the models' performance in terms of identifying post-tranlational modifications (PTMs), efficiency and robustness to peptide length, noise peaks and missing fragment ratio, which are important influencing factors while seldom be considered. Leveraging this benchmark, we conduct a large-scale study of current methods, report many insightful findings that open up new possibilities for future development. The benchmark is open-sourced to facilitate future research and application. The code is available at \url{https: //github. com/Westlake-OmicsAI/NovoBench}.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

VMamba: Visual State Space Model

  • Yue Liu
  • Yunjie Tian
  • Yuzhong Zhao
  • Hongtian Yu
  • Lingxi Xie
  • Yaowei Wang
  • Qixiang Ye
  • Jianbin Jiao

Designing computationally efficient network architectures remains an ongoing necessity in computer vision. In this paper, we adapt Mamba, a state-space language model, into VMamba, a vision backbone with linear time complexity. At the core of VMamba is a stack of Visual State-Space (VSS) blocks with the 2D Selective Scan (SS2D) module. By traversing along four scanning routes, SS2D bridges the gap between the ordered nature of 1D selective scan and the non-sequential structure of 2D vision data, which facilitates the collection of contextual information from various sources and perspectives. Based on the VSS blocks, we develop a family of VMamba architectures and accelerate them through a succession of architectural and implementation enhancements. Extensive experiments demonstrate VMamba’s promising performance across diverse visual perception tasks, highlighting its superior input scaling efficiency compared to existing benchmark models. Source code is available at https: //github. com/MzeroMiko/VMamba

PDF Details DOI

AAAI Conference 2023 Conference Paper

Cluster-Guided Contrastive Graph Clustering Network

  • Xihong Yang
  • Yue Liu
  • Sihang Zhou
  • Siwei Wang
  • Wenxuan Tu
  • Qun Zheng
  • Xinwang Liu
  • Liming Fang

Benefiting from the intrinsic supervision information exploitation capability, contrastive learning has achieved promising performance in the field of deep graph clustering recently. However, we observe that two drawbacks of the positive and negative sample construction mechanisms limit the performance of existing algorithms from further improvement. 1) The quality of positive samples heavily depends on the carefully designed data augmentations, while inappropriate data augmentations would easily lead to the semantic drift and indiscriminative positive samples. 2) The constructed negative samples are not reliable for ignoring important clustering information. To solve these problems, we propose a Cluster-guided Contrastive deep Graph Clustering network (CCGC) by mining the intrinsic supervision information in the high-confidence clustering results. Specifically, instead of conducting complex node or edge perturbation, we construct two views of the graph by designing special Siamese encoders whose weights are not shared between the sibling sub-networks. Then, guided by the high-confidence clustering information, we carefully select and construct the positive samples from the same high-confidence cluster in two views. Moreover, to construct semantic meaningful negative sample pairs, we regard the centers of different high-confidence clusters as negative samples, thus improving the discriminative capability and reliability of the constructed sample pairs. Lastly, we design an objective function to pull close the samples from the same cluster while pushing away those from other clusters by maximizing and minimizing the cross-view cosine similarity between positive and negative samples. Extensive experimental results on six datasets demonstrate the effectiveness of CCGC compared with the existing state-of-the-art algorithms. The code of CCGC is available at https://github.com/xihongyang1999/CCGC on Github.

PDF Details DOI

TIST Journal 2023 Journal Article

Discovering Causes of Traffic Congestion via Deep Transfer Clustering

  • Mudan Wang
  • Yuan Yuan
  • Huan Yan
  • HONGJIE SUI
  • Fan Zuo
  • Yue Liu
  • Yong Li
  • Depeng Jin

Traffic congestion incurs long delay in travel time, which seriously affects our daily travel experiences. Exploring why traffic congestion occurs is significantly important to effectively address the problem of traffic congestion and improve user experience. Traditional approaches to mine the congestion causes depend on human efforts, which is time consuming and cost-intensive. Hence, we aim at discovering the known and unknown causes of traffic congestion in a systematic way. However, to achieve it, there are three challenges: (1) traffic congestion is affected by several factors with complex spatio-temporal relations; (2) there are a few samples of congestion data with known causes due to the limitation of human label; (3) more unknown congestion causes are unexplored since several factors contribute to traffic congestion. To address above challenges, we design a congestion cause discovery system consisting of two modules: (1) congestion feature extraction module, which extracts the important features distinguishing between different causes of congestion; and (2) congestion cause discovery module, which designs a deep semi-supervised learning based framework to discover the causes of traffic congestion with limited labeled data. Specifically, in pre-training stage, it first leverages a few labeled data as prior knowledge to pre-train the model. Then, in clustering stage, we propose two different clustering methods to discover the congestion causes. For the first clustering method, we extend the classic deep embedded clustering model to produce clusters via soft assignment. For the second one, we iteratively use k -means to group the latent features extracted from the pre-trained model, and use the cluster results as pseudo-labels to fine-tune the network. Extensive experiments show that the performance of our methods is superior to the state-of-the-art baselines, which demonstrates the effectiveness of the proposed cause discovery system. Additionally, our system is deployed and used in the practical production environment at Amap.

Details DOI

EAAI Journal 2023 Journal Article

Event-driven spiking neural network based on membrane potential modulation for remote sensing image classification

  • Li-Ye Niu
  • Ying Wei
  • Yue Liu

Spiking neural network (SNN) based on sparse triggering and event-driven is a hardware-friendly model. SNN can provide an ultra-low power alternative for the deep neural network (DNN) to process remote sensing images. Brain information processing depends on the action potential of neurons. Therefore, the biological rationality of the artificial neural network (ANN) has been questioned. SNN is a more suitable model for brain information processing mechanisms. At present, the SNN obtained by ANN conversion has achieved the best performance in the current image processing tasks. However, the method based on ANN to SNN will have performance loss in the conversion process. Herein, we proposed a spiking neuron threshold-following reset (TF-reset) method and a membrane potential modulation method to reduce the loss of network conversion. We theoretically analyzed the proposed TF-reset and deduced the relationship between spike firing rate and neuron activation. In the experiment, we used an improved VGG-15 architecture combined with the method of transfer learning to apply the model to the classification task of remote sensing images. SNN-VGG-15 based on TF-reset and membrane potential modulation algorithm achieved a classification accuracy of 99. 14%, 94. 54%, and 95. 00% on UCM, RSSCN7, and AID. Our algorithm can not only realize the lossless conversion of SNN but also outperforms the original network in classification performance on UCM and RSSCN7. In addition, our model also has advantages in energy consumption and noise robustness. The algorithm in this paper can provide a reference for the research remote sensing images procession using SNN.

Details DOI

EAAI Journal 2023 Journal Article

Few-shot person re-identification based on Feature Set Augmentation and Metric Fusion

  • Guizhen Chen
  • Guofeng Zou
  • Yue Liu
  • Xiaofei Zhang
  • Guixia Fu

Person re-identification identifies pedestrians by analyzing image information from surveillance videos. However, it faces challenges like occlusion, changing lighting, and costly annotation. Thus, it is often performed in a few-shot environment with limited images. In response to the problem of insufficient available pedestrian images in person re-identification, a few-shot person re-identification method based on feature set augmentation and metric fusion is proposed. In this work, firstly, a feature augmentation method is introduced into the feature embedding module. This method introduces multi-head self-attention in different feature extraction layers and spatial attention in feature fusion of different feature extraction layers, which can extract more diverse and discriminative pedestrian features. Secondly, a dual metric method combining Euclidean and cosine distance is proposed in the metric module to comprehensively measure the absolute spatial distance and directional difference of pedestrian features. In this way, the reliability of pedestrian similarity measurement is improved. Then, pedestrian feature similarity scores are obtained separately using the dual metric and relation metric methods. Finally, the combined metric score is obtained by weighted fusion, and the combined metric score is used to construct the joint loss to realize the overall optimization and training of the network. Experimental results on three small datasets, Market-mini, Duke-mini, and MSMT17-mini, show that the proposed method significantly improves recognition performance compared to other few-shot learning algorithms. Specifically, in scenarios 5-way 1-shot and 5-way 5-shot, the average recognition accuracies are 92. 54% and 96. 99%, 87. 93% and 96. 08%, and 71. 68% and 84. 51%, respectively.

Details DOI

AAAI Conference 2023 Conference Paper

Graph Anomaly Detection via Multi-Scale Contrastive Learning Networks with Augmented View

  • Jingcan Duan
  • Siwei Wang
  • Pei Zhang
  • En Zhu
  • Jingtao Hu
  • Hu Jin
  • Yue Liu
  • Zhibin Dong

Graph anomaly detection (GAD) is a vital task in graph-based machine learning and has been widely applied in many real-world applications. The primary goal of GAD is to capture anomalous nodes from graph datasets, which evidently deviate from the majority of nodes. Recent methods have paid attention to various scales of contrastive strategies for GAD, i.e., node-subgraph and node-node contrasts. However, they neglect the subgraph-subgraph comparison information which the normal and abnormal subgraph pairs behave differently in terms of embeddings and structures in GAD, resulting in sub-optimal task performance. In this paper, we fulfill the above idea in the proposed multi-view multi-scale contrastive learning framework with subgraph-subgraph contrast for the first practice. To be specific, we regard the original input graph as the first view and generate the second view by graph augmentation with edge modifications. With the guidance of maximizing the similarity of the subgraph pairs, the proposed subgraph-subgraph contrast contributes to more robust subgraph embeddings despite of the structure variation. Moreover, the introduced subgraph-subgraph contrast cooperates well with the widely-adopted node-subgraph and node-node contrastive counterparts for mutual GAD performance promotions. Besides, we also conduct sufficient experiments to investigate the impact of different graph augmentation approaches on detection performance. The comprehensive experimental results well demonstrate the superiority of our method compared with the state-of-the-art approaches and the effectiveness of the multi-view subgraph pair contrastive strategy for the GAD task. The source code is released at https://github.com/FelixDJC/GRADATE.

PDF Details DOI

AAAI Conference 2023 Conference Paper

Hard Sample Aware Network for Contrastive Deep Graph Clustering

  • Yue Liu
  • Xihong Yang
  • Sihang Zhou
  • Xinwang Liu
  • Zhen Wang
  • Ke Liang
  • Wenxuan Tu
  • Liang Li

Contrastive deep graph clustering, which aims to divide nodes into disjoint groups via contrastive mechanisms, is a challenging research spot. Among the recent works, hard sample mining-based algorithms have achieved great attention for their promising performance. However, we find that the existing hard sample mining methods have two problems as follows. 1) In the hardness measurement, the important structural information is overlooked for similarity calculation, degrading the representativeness of the selected hard negative samples. 2) Previous works merely focus on the hard negative sample pairs while neglecting the hard positive sample pairs. Nevertheless, samples within the same cluster but with low similarity should also be carefully learned. To solve the problems, we propose a novel contrastive deep graph clustering method dubbed Hard Sample Aware Network (HSAN) by introducing a comprehensive similarity measure criterion and a general dynamic sample weighing strategy. Concretely, in our algorithm, the similarities between samples are calculated by considering both the attribute embeddings and the structure embeddings, better revealing sample relationships and assisting hardness measurement. Moreover, under the guidance of the carefully collected high-confidence clustering information, our proposed weight modulating function will first recognize the positive and negative samples and then dynamically up-weight the hard sample pairs while down-weighting the easy ones. In this way, our method can mine not only the hard negative samples but also the hard positive sample, thus improving the discriminative capability of the samples further. Extensive experiments and analyses demonstrate the superiority and effectiveness of our proposed method. The source code of HSAN is shared at https://github.com/yueliu1999/HSAN and a collection (papers, codes and, datasets) of deep graph clustering is shared at https://github.com/yueliu1999/Awesome-Deep-Graph-Clustering on Github.

PDF Details DOI

ICML Conference 2023 Conference Paper

Trustworthy Policy Learning under the Counterfactual No-Harm Criterion

  • Haoxuan Li 0001
  • Chunyuan Zheng 0001
  • Yixiao Cao
  • Zhi Geng
  • Yue Liu
  • Peng Wu 0012

Trustworthy policy learning has significant importance in making reliable and harmless treatment decisions for individuals. Previous policy learning approaches aim at the well-being of subgroups by maximizing the utility function (e. g. , conditional average causal effects, post-view click-through&conversion rate in recommendations), however, individual-level counterfactual no-harm criterion has rarely been discussed. In this paper, we first formalize the counterfactual no-harm criterion for policy learning from a principal stratification perspective. Next, we propose a novel upper bound for the fraction negatively affected by the policy and show the consistency and asymptotic normality of the estimator. Based on the estimators for the policy utility and harm upper bounds, we further propose a policy learning approach that satisfies the counterfactual no-harm criterion, and prove its consistency to the optimal policy reward for parametric and non-parametric policy classes, respectively. Extensive experiments are conducted to show the effectiveness of the proposed policy learning approach for satisfying the counterfactual no-harm criterion.

Details

NeurIPS Conference 2023 Conference Paper

Understanding the Limitations of Deep Models for Molecular property prediction: Insights and Solutions

  • Jun Xia
  • Lecheng Zhang
  • Xiao Zhu
  • Yue Liu
  • Zhangyang Gao
  • Bozhen Hu
  • Cheng Tan
  • Jiangbin Zheng

Molecular Property Prediction (MPP) is a crucial task in the AI-driven Drug Discovery (AIDD) pipeline, which has recently gained considerable attention thanks to advancements in deep learning. However, recent research has revealed that deep models struggle to beat traditional non-deep ones on MPP. In this study, we benchmark 12 representative models (3 non-deep models and 9 deep models) on 15 molecule datasets. Through the most comprehensive study to date, we make the following key observations: \textbf{(\romannumeral 1)} Deep models are generally unable to outperform non-deep ones; \textbf{(\romannumeral 2)} The failure of deep models on MPP cannot be solely attributed to the small size of molecular datasets; \textbf{(\romannumeral 3)} In particular, some traditional models including XGB and RF that use molecular fingerprints as inputs tend to perform better than other competitors. Furthermore, we conduct extensive empirical investigations into the unique patterns of molecule data and inductive biases of various models underlying these phenomena. These findings stimulate us to develop a simple-yet-effective feature mapping method for molecule data prior to feeding them into deep models. Empirically, deep models equipped with this mapping method can beat non-deep ones in most MoleculeNet datasets. Notably, the effectiveness is further corroborated by extensive experiments on cutting-edge dataset related to COVID-19 and activity cliff datasets.

PDF Details

AIJ Journal 2022 Journal Article

A local method for identifying causal relations under Markov equivalence

  • Zhuangyan Fang
  • Yue Liu
  • Zhi Geng
  • Shengyu Zhu
  • Yangbo He

Causality is important for designing interpretable and robust methods in artificial intelligence research. We propose a local approach to identify whether a variable is a cause of a given target under the framework of causal graphical models of directed acyclic graphs (DAGs). In general, the causal relation between two variables may not be identifiable from observational data as many causal DAGs encoding different causal relations are Markov equivalent. In this paper, we first introduce a sufficient and necessary graphical condition to check the existence of a causal path from a variable to a target in every Markov equivalent DAG. Next, we provide local criteria for identifying whether a variable is a cause/non-cause of a target based only on the local structure instead of the entire graph. Finally, we propose a local learning algorithm for this causal query via learning the local structure of the variable and some additional statistical independence tests related to the target. Simulation studies show that our local algorithm is efficient and effective, compared with other state-of-art methods.

Details DOI

AAAI Conference 2022 Conference Paper

Deep Graph Clustering via Dual Correlation Reduction

  • Yue Liu
  • Wenxuan Tu
  • Sihang Zhou
  • Xinwang Liu
  • Linxuan Song
  • Xihong Yang
  • En Zhu

Deep graph clustering, which aims to reveal the underlying graph structure and divide the nodes into different groups, has attracted intensive attention in recent years. However, we observe that, in the process of node encoding, existing methods suffer from representation collapse which tends to map all data into the same representation. Consequently, the discriminative capability of the node representation is limited, leading to unsatisfied clustering performance. To address this issue, we propose a novel self-supervised deep graph clustering method termed Dual Correlation Reduction Network (DCRN) by reducing information correlation in a dual manner. Specifically, in our method, we first design a siamese network to encode samples. Then by forcing the cross-view sample correlation matrix and cross-view feature correlation matrix to approximate two identity matrices, respectively, we reduce the information correlation in the dual-level, thus improving the discriminative capability of the resulting features. Moreover, in order to alleviate representation collapse caused by over-smoothing in GCN, we introduce a propagation regularization term to enable the network to gain long-distance information with the shallow network structure. Extensive experimental results on six benchmark datasets demonstrate the effectiveness of the proposed DCRN against the existing state-of-the-art methods. The code of DCRN is available at https: //github. com/yueliu1999/DCRN and a collection (papers, codes and, datasets) of deep graph clustering is shared at https: //github. com/yueliu1999/Awesome-Deep- Graph-Clustering on Github.

PDF Details

IJCAI Conference 2022 Conference Paper

Initializing Then Refining: A Simple Graph Attribute Imputation Network

  • Wenxuan Tu
  • Sihang Zhou
  • Xinwang Liu
  • Yue Liu
  • Zhiping Cai
  • En Zhu
  • Changwang Zhang
  • Jieren Cheng

Representation learning on the attribute-missing graphs, whose connection information is complete while the attribute information of some nodes is missing, is an important yet challenging task. To impute the missing attributes, existing methods isolate the learning processes of attribute and structure information embeddings, and force both resultant representations to align with a common in-discriminative normal distribution, leading to inaccurate imputation. To tackle these issues, we propose a novel graph-oriented imputation framework called initializing then refining (ITR), where we first employ the structure information for initial imputation, and then leverage observed attribute and structure information to adaptively refine the imputed latent variables. Specifically, we first adopt the structure embeddings of attribute-missing samples as the embedding initialization, and then refine these initial values by aggregating the reliable and informative embeddings of attribute-observed samples according to the affinity structure. Specially, in our refining process, the affinity structure is adaptively updated through iterations by calculating the sample-wise correlations upon the recomposed embeddings. Extensive experiments on four benchmark datasets verify the superiority of ITR against state-of-the-art methods.

PDF Details DOI

JBHI Journal 2022 Journal Article

Investigate the Neuro Mechanisms of Stereoscopic Visual Fatigue

  • Kang Yue
  • Mei Guo
  • Yue Liu
  • Haochen Hu
  • Kai Lu
  • Shanshan Chen
  • Danli Wang

Stereoscopic visual fatigue (SVF) due to prolonged immersion in the virtual environment can lead to negative user experience, thus hindering the development of virtual reality (VR) industry. Previous studies have focused on investigating the evaluation indicators associated with SVF, while few studies have been conducted to reveal the underlying neural mechanism, especially in VR applications. In this paper, a modified Go/NoGo paradigm was adopted to induce SVF in VR environment with Go trials for maintaining participants’ attention and NoGo trials for investigating the neural effects under SVF. Random dot stereograms (RDSs) with 11 disparities were presented to evoke the depth-related visual evoked potentials (DVEPs) during 64-channel EEG recordings. EEG datasets collected from 15 participants in NoGo trials were selected to conduct individual processing and group analysis, in which the characteristics of the DVEPs components for various fatigue degrees were compared and independent components were clustered to explore the original cortex areas related to SVF. Point-by-point permutation statistics revealed that DVEPs sample points from 230 ms to 280 ms (component P2) in most brain areas changed significantly when SVF increased. Additionally, independent component analysis (ICA) identified that component P2 which originated from posterior cingulate cortex and precuneus, was associated statistically with SVF. We believe that SVF is rather a conscious status concerning the changes of self-awareness or self-location awareness than the performance reduction of retinal image processing. Moreover, we suggest that indicators representing higher conscious state may be a better indicator for SVF evaluation in VR environments.

Details DOI

NeurIPS Conference 2022 Conference Paper

Module-Aware Optimization for Auxiliary Learning

  • Hong Chen
  • Xin Wang
  • Yue Liu
  • Yuwei Zhou
  • Chaoyu Guan
  • Wenwu Zhu

Auxiliary learning is a widely adopted practice in deep learning, which aims to improve the model performance on the primary task by exploiting the beneficial information in the auxiliary loss. Existing auxiliary learning methods only focus on balancing the auxiliary loss and the primary loss, ignoring the module-level auxiliary influence, i. e. , an auxiliary loss will be beneficial for optimizing specific modules within the model but harmful to others, failing to make full use of auxiliary information. To tackle the problem, we propose a Module-Aware Optimization approach for Auxiliary Learning (MAOAL). The proposed approach considers the module-level influence through the learnable module-level auxiliary importance, i. e. , the importance of each auxiliary loss to each module. Specifically, the proposed approach jointly optimizes the module-level auxiliary importance and the model parameters in a bi-level manner. In the lower optimization, the model parameters are optimized with the importance parameterized gradient, while in the upper optimization, the module-level auxiliary importance is updated with the implicit gradient from a small developing dataset. Extensive experiments show that our proposed MAOAL method consistently outperforms state-of-the-art baselines for different auxiliary losses on various datasets, demonstrating that our method can serve as a powerful generic tool for auxiliary learning.

PDF Details

JBHI Journal 2022 Journal Article

Sparse-Based Domain Adaptation Network for OCTA Image Super-Resolution Reconstruction

  • Huaying Hao
  • Cong Xu
  • Dan Zhang
  • Qifeng Yan
  • Jiong Zhang
  • Yue Liu
  • Yitian Zhao

Retinal Optical Coherence Tomography Angiography (OCTA) with high-resolution is important for the quantification and analysis of retinal vasculature. However, the resolution of OCTA images is inversely proportional to the field of view at the same sampling frequency, which is not conducive to clinicians for analyzing larger vascular areas. In this paper, we propose a novel S parse-based domain A daptation S uper- R esolution network (SASR) for the reconstruction of realistic $6\times \text{6}{\rm{mm}}^{2}$ /low-resolution (LR) OCTA images to high-resolution (HR) representations. To be more specific, we first perform a simple degradation of the $3\times \text{3}\, {\rm{mm}}^{2}$ /high-resolution (HR) image to obtain the synthetic LR image. An efficient registration method is then employed to register the synthetic LR with its corresponding $3\times \text{3}\, {\rm{mm}}^{2}$ image region within the $6\times \text{6}\, {\rm{mm}}^{2}$ image to obtain the cropped realistic LR image. We then propose a multi-level super-resolution model for the fully-supervised reconstruction of the synthetic data, guiding the reconstruction of the realistic LR images through a generative-adversarial strategy that allows the synthetic and realistic LR images to be unified in the feature domain. Finally, a novel sparse edge-aware loss is designed to dynamically optimize the vessel edge structure. Extensive experiments on two OCTA sets have shown that our method performs better than state-of-the-art super-resolution reconstruction methods. In addition, we have investigated the performance of the reconstruction results on retina structure segmentations, which further validate the effectiveness of our approach.

Details DOI

NeurIPS Conference 2021 Conference Paper

CSAW-M: An Ordinal Classification Dataset for Benchmarking Mammographic Masking of Cancer

  • Moein Sorkhei
  • Yue Liu
  • Hossein Azizpour
  • Edward Azavedo
  • Karin Dembrower
  • Dimitra Ntoula
  • Athanasios Zouzos
  • Fredrik Strand

Interval and large invasive breast cancers, which are associated with worse prognosis than other cancers, are usually detected at a late stage due to false negative assessments of screening mammograms. The missed screening-time detection is commonly caused by the tumor being obscured by its surrounding breast tissues, a phenomenon called masking. To study and benchmark mammographic masking of cancer, in this work we introduce CSAW-M, the largest public mammographic dataset, collected from over 10, 000 individuals and annotated with potential masking. In contrast to the previous approaches which measure breast image density as a proxy, our dataset directly provides annotations of masking potential assessments from five specialists. We also trained deep learning models on CSAW-M to estimate the masking level and showed that the estimated masking is significantly more predictive of screening participants diagnosed with interval and large invasive cancers -- without being explicitly trained for these tasks -- than its breast density counterparts.

PDF Details

ICML Conference 2020 Conference Paper

Adding seemingly uninformative labels helps in low data regimes

  • Christos Matsoukas
  • Albert Bou I Hernandez
  • Yue Liu
  • Karin Dembrower
  • Gisele Miranda
  • Emir Konuk
  • Johan Fredin Haslum
  • Athanasios Zouzos

Evidence suggests that networks trained on large datasets generalize well not solely because of the numerous training examples, but also class diversity which encourages learning of enriched features. This raises the question of whether this remains true when data is scarce - is there an advantage to learning with additional labels in low-data regimes? In this work, we consider a task that requires difficult-to-obtain expert annotations: tumor segmentation in mammography images. We show that, in low-data settings, performance can be improved by complementing the expert annotations with seemingly uninformative labels from non-expert annotators, turning the task into a multi-class problem. We reveal that these gains increase when less expert data is available, and uncover several interesting properties through further studies. We demonstrate our findings on CSAW-S, a new dataset that we introduce here, and confirm them on two public datasets.

Details

UAI Conference 2020 Conference Paper

Collapsible IDA: Collapsing Parental Sets for Locally Estimating Possible Causal Effects

  • Yue Liu
  • Zhuangyan Fang
  • Yangbo He
  • Zhi Geng

It is clear that some causal effects cannot be identified from observational data when the causal directed acyclic graph is absent. In such cases, IDA is a useful framework which estimates all possible causal effects by adjusting for all possible parental sets. In this paper, we combine the adjustment set selection procedure with the original IDA framework. Our goal is to find a common set that can be subtracted from all possible parental sets without influencing the back-door adjustment. To this end, we first introduce graphical conditions to decide whether a treatment’s neighbor or parent in a completed partially directed acyclic graph (CPDAG) can be subtracted and then provide a procedure to construct a subtractable set from those subtractable vertices. We next combine the procedure with the IDA framework and provide a fully local modification of IDA. Experimental results show that, with our modification, both the number of possible parental sets and the size of each possible parental set enumerated by the modified IDA decrease, making it possible to estimate all possible causal effects more efficiently.

Details

JMLR Journal 2020 Journal Article

Local Causal Network Learning for Finding Pairs of Total and Direct Effects

  • Yue Liu
  • Zhuangyan Fang
  • Yangbo He
  • Zhi Geng
  • Chunchen Liu

In observational studies, it is important to evaluate not only the total effect but also the direct and indirect effects of a treatment variable on a response variable. In terms of local structural learning of causal networks, we try to find all possible pairs of total and direct causal effects, which can further be used to calculate indirect causal effects. An intuitive global learning approach is first to find an essential graph over all variables representing all Markov equivalent causal networks, and then enumerate all equivalent networks and estimate a pair of the total and direct effects for each of them. However, it could be inefficient to learn an essential graph and enumerate equivalent networks when the true causal graph is large. In this paper, we propose a local learning approach instead. In the local learning approach, we first learn locally a chain component containing the treatment. Then, if necessary, we learn locally a chain component containing the response. Next, we locally enumerate all possible pairs of the treatment's parents and the response's parents. Finally based on these pairs, we find all possible pairs of total and direct effects of the treatment on the response. [abs] [ pdf ][ bib ] &copy JMLR 2020. ( edit, beta )

PDF Details

YNIMG Journal 2019 Journal Article

Detection of neural connections with ex vivo MRI using a ferritin-encoding trans-synaptic virus

  • Ning Zheng
  • Peng Su
  • Yue Liu
  • Huadong Wang
  • Binbin Nie
  • Xiaohui Fang
  • Yue Xu
  • Kunzhang Lin

The elucidation of neural networks is essential to understanding the mechanisms of brain functions and brain disorders. Neurotropic virus-based trans-synaptic tracing tools have become an effective method for dissecting the structure and analyzing the function of neural-circuitry. However, these tracing systems rely on fluorescent signals, making it hard to visualize the panorama of the labeled networks in mammalian brain in vivo. One MRI method, Diffusion Tensor Imaging (DTI), is capable of imaging the networks of the whole brain in live animals but without information of anatomical connections through synapses. In this report, a chimeric gene coding for ferritin and enhanced green fluorescent protein (EGFP) was integrated into Vesicular stomatitis virus (VSV), a neurotropic virus that is able to spread anterogradely in synaptically connected networks. After the animal was injected with the recombinant VSV (rVSV), rVSV-Ferritin-EGFP, into the somatosensory cortex (SC) for four days, the labeled neural-network was visualized in the postmortem whole brain with a T2-weighted MRI sequence. The modified virus transmitted from SC to synaptically connected downstream regions. The results demonstrate that rVSV-Ferritin-EGFP could be used as a bimodal imaging vector for detecting synaptically connected neural-network with both ex vivo MRI and fluorescent imaging. The strategy in the current study has the potential to longitudinally monitor the global structure of a given neural-network in living animals.

Details DOI

TIST Journal 2019 Journal Article

Local Learning Approaches for Finding Effects of a Specified Cause and Their Causal Paths

  • Yue Liu
  • Zheng Cai
  • Chunchen Liu
  • Zhi Geng

Causal networks are used to describe and to discover causal relationships among variables and data generating mechanisms. There have been many approaches for learning a global causal network of all observed variables. In many applications, we may be interested in finding what are the effects of a specified cause variable and what are the causal paths from the cause variable to its effects. Instead of learning a global causal network, we propose several local learning approaches for finding all effects (or descendants) of the specified cause variable and the causal paths from the cause variable to some effect variable of interest. We discuss the identifiability of the effects and the causal paths from observed data and prior knowledge. For the case that the causal paths are not identifiable, our approaches try to find a path set that contains the causal paths of interest.

Details DOI

IJCAI Conference 2018 Conference Paper

FISH-MML: Fisher-HSIC Multi-View Metric Learning

  • Changqing Zhang
  • Yeqinq Liu
  • Yue Liu
  • Qinghua Hu
  • Xinwang Liu
  • Pengfei Zhu

This work presents a simple yet effective model for multi-view metric learning, which aims to improve the classification of data with multiple views, e. g. , multiple modalities or multiple types of features. The intrinsic correlation, different views describing same set of instances, makes it possible and necessary to jointly learn multiple metrics of different views, accordingly, we propose a multi-view metric learning method based on Fisher discriminant analysis (FDA) and Hilbert-Schmidt Independence Criteria (HSIC), termed as Fisher-HSIC Multi-View Metric Learning (FISH-MML). In our approach, the class separability is enforced in the spirit of FDA within each single view, while the consistence among different views is enhanced based on HSIC. Accordingly, both intra-view class separability and inter-view correlation are well addressed in a unified framework. The learned metrics can improve multi-view classification, and experimental results on real-world datasets demonstrate the effectiveness of the proposed method.

PDF Details